How to test usability

Good usability means that real people can do real tasks in a reasonable amount of time. You'd like to think there's one true way to test usability, but the method you use to examine the usability of an information product depends on what it is, what "stage" of development it's at, and how people use it.

Two popular forms of usability testing are paper prototype tests for incomplete mockups of a new system, and lab usability tests for more complete information products.

Paper prototype testing

If you have a product that's still in the design phase, such as the concept or mockup for a new information product such as a website or user interface, you can perform a paper prototype test. In a paper prototype test, you start with a mockup of a new design, then you print it on paper.

To test the new design, you craft scenario tasks that prompt the user with questions like "What would you click on to do (task)?" Give the tester the scenario task, and present them with the starting point for the mockup, and respond to them as though you were the "system" being tested.

For example, to test a new website design, you might have a visual mockup of the new design. The design concept might only exist as an image, not even coded as a temporary website. In a usability test, you place a mockup of the front page in front of your tester, and give your tester a scenario task such as:

You want to learn what this organization does and where they operate. What would you click on to get more information?

If the website mockup included an "About us" link, such as in the header or the footer, the tester would tap on it. For the purposes of a paper prototype test, that might be sufficient to know that the design adequately guided the user to the "About" page.

For software designs, you might have mockups of "sub-pages" too. One example might be a "Settings" app that has pages or dialog boxes for groups of related settings: such as one group of "Display" settings, another group for "Sound" settings, another for "Network," and so on. As you perform the paper prototype test, if you have mockups for these sub-pages, present them to the tester as they "tap" on each item in the Settings app.

Lab usability tests

Many usability tests are still best done by watching someone attempt to do real work with the new design. These types of "traditional" or lab usability tests might be performed in-person or remotely, such as using Zoom or another online meeting tool.

In a lab usability test, you typically use a full working version of the new system. This might not be feature-complete, but enough of a working model that testers will be able to use it effectively. You might customize the scenario tasks used to test such a system to narrow the focus on the areas you know are complete. This isn't a guarantee that testers won't wander into unfinished parts of the design, but most testers should be able to exercise the parts of the design that need testing.

To perform this kind of usability test, you present your tester with a series of scenario tasks, and observe them while they attempt to perform the tasks. These need to be realistic scenario tasks, that sets a brief context then asks the tester to do something specific. This is a step further than the prototype test, which only asks testers to indicate what they would click on to achieve a result; by contrast, the lab usability test should ask testers to complete the action:

You want to learn what this organization does and where they operate. Find a page that has more information about the X organization.

If the website included an "About us" page, the goal of the scenario task would be that the tester navigates to the "About us" page.

Fit the test to the product

There is no usability test that is "one size fits all." If you are testing a new version of a product, and you don't have a working version of the new system, your best option is likely the paper prototype test. For a new design that is more complete, such as an update to an existing system or a new system that has spent more time in development, you might instead choose the more traditional "lab" usability test.