A Playtesting Primer

Given the previous benefits for doing usability playtesting, you might wonder why some studios, or more likely, the majority of studios, choose not to do it. There are quite a wide range of reasons for not doing usability testing, some easy to solve, others less so.

• No awareness or incorrect awareness - some studios are not simply aware of usability playtesting, it's not part of their discussions, and certainly not in their roadmap. Possibly worse are those who think they know what usability playtesting is, but their understanding is wrong.

• It's not a requirement - It's possible to release a game without doing any usability playtesting. If budgets, or time, are tight, then some studios will either not be able to do usability playtesting, or choose not to do it.

• Ego - some team members simply may not want any feedback telling them that their features or systems don’t work as intended. This is a major red flag indicator for a team which is not likely to perform well, even one person with this attitude can spoil the learning potential for an entire team.

• Perceived experience - some team members may say that they’ve been doing game development for many years and that they don’t need their games tested for usability. There are multiple issues here, firstly some confuse years in the job (quantity of experience) with how good they are at their job (quality of their experience). However, repeating the same bad practices for years on end does not make you experienced, or good, at your job. Secondly, just because you have experience in a job role, does not mean that you predict how an audience will interact with your game. This brings us to the next point.

• “We’re gamers too” - some team members may think that they know their audience because they themselves are gamers too. However, the development team are not the audience. In addition, as development on the game progresses, the team will become overly familiar with its systems and how they work, and they will stop experiencing the flaws which new players almost certainly will.

• Perceived value - some studios feel that it’s just not worth the effort. What they often mean here is that the issues found are small, too small in relation to the effort required to find them. I would often hear some developers say ‘Usability testing is so important’, but they would never actually do it. Their behaviour shows that it’s not that important, to them at least, there are other factors on their mind to which they attribute higher value.

• Low numbers of participants - some studios just cannot come to terms with a low number of players being used for usability playtesting (6-12 is quite common). They are aiming for an audience of thousands or even millions, surely testing with 6 players cannot be enough.

• Myths - there are many myths around usability playtesting, and one of the more common ones is that it dumbs-down the game and makes it easier. This is not true of course, you may indeed want the gameplay to be difficult, but no one wants the game’s menus, UI, or controls to be difficult to use. There is a difference between making the game easier to understand and use and making the gameplay easier.

To a large extent, the above issues can be addressed through awareness and education, it's possible that the team just are not aware of what usability playtesting is and how it helps. They may think they know, but that thinking is incorrect. For this reason, I would consider awareness and education to be the most fundamental starting point in getting a team to do usability playtesting. If the team don't know what it does or how it'll help them, or if their current understanding is incorrect, then it's going to be very difficult to ever get them to do it. Luckily this can be overcome, usability testing and its benefits can be explained, however education by itself is not enough to get studios to do it.

Assuming studios are aware of usability testing and also understand its benefits to game design, there are two further barriers to entry:

• Process Expertise - does someone on the team have enough expertise in how to design, run, and analyse a usability playtest? Can the rest of the team trust the results when presented, i.e. any issues found from the playtest must be due to issues with the game's design, not flaws with how the playtest process was executed. It's not always the case that usability playtests are conducted with enough rigour, staff can be incompetent and other team members learn to not trust their work, or override the results from the playtests with their own intuition. The point here is that just because a studio is doing usability playtests, does not mean that they are doing them well. Finding suitable staff who can run a usability playtest is not straightforward, which relates to the next point.
• Budget - as specialist staff who can run usability playtests are in short supply, they can often command high salaries. Relating to the high salary is utilisation, how often would this person be needed? For some studios, they may only want playtesting done at certain points in development, so it wouldn't make sense for them to hire full time staff. Financially, it just doesn't make sense for some studios to have dedicated staff to run usability testing.

There are quite a few barriers to entry then, which helps explain why if you consider all the games studios worldwide, the percentage of them which have dedicated (and qualified) staff for usability playtesting would be very low indeed. Correspondingly, the number of games released each year which have gone through formal usability playtesting is very low. The outcome of this situation is that the vast majority of games are released will many usability issues present. It's possible, even likely, than many of these usability issues will be small annoyances, possible for the player to overcome, however others may not be, and could ruin the player experience.