The art of writing scenario tasks
A good scenario task sets a brief context then asks the tester to do something specific.
Scenario tasks are the heart of any usability test. In a usability test, you give your testers a series of scenario tasks and then observe the tester in what they do, what they click on, and what they say.
A good scenario task sets a brief context then asks the tester to do something specific. But when we write these scenario tasks, it's important not to reuse any keywords from the system being tested. Using a common word might be okay, depending on the system, but in general you want to leverage your thesaurus when writing these scenario tasks to avoid reusing any words or phrases that the tester might see in the design they are testing.
For example, if you were testing a website and wanted to examine if testers could navigate to the "About us" page, you would not want to present the tester with a scenario task that repeated the phrase "About us" while setting the context or presenting the task goal. This is a "do not" example of such a scenario task:
You want to work with X organization, but you want to learn more about them. Find a web page that describes what they are about.
The repeated use of "about" seems odd in this scenario task. Most testers will recognize this as a hint that they should find a web page with "about" in the title. The scenario task then becomes a "hunt" exercise where the tester is just looking for pages with "about" in the title, until they find the "About us" page.
Writing scenario tasks that set the context and define the goal without repeating keywords from the system is sometimes more of an art than a science. It's a skill you develop with practice. I also wanted to share several scenario tasks that I have used in previous usability tests, to demonstrate scenario tasks that set a brief context then ask the tester to do something specific.
A text editor
Several years ago, I mentored an intern for usability testing in the GNOME Project. If you aren't familiar with this project, GNOME is an open source desktop; just like MacOS is a desktop and Windows is a desktop, GNOME is a desktop for computers running Linux.
GNOME includes a host of useful apps, including a text editor. At the time, the default text editor was called "Gedit" ("GNOME edit"). The GNOME Design team indicated that many people used Gedit to write brief notes or drafts of other documents, so we wrote these scenario tasks to exercise Gedit in this way:
1. You need to type up a quick note for yourself, briefly describing an event that you want to remember later. You start the Gedit text editor (this has been done for you). Please type the following short paragraphs into the text editor:
Note for myself:
Jenny and I decided to throw a surprise party for Jeff, for Jeff's birthday. We'll get together at 5:30 on Friday after work, at Jeff's place. Jenny arranged for Jeff's roommate to keep him away until 7:00.
We need to get the decorations up and music ready to go by 6:30, when people will start to arrive. We asked everyone to be there no later than 6:45.
Save this note as party reminder.txt in the Documents folder.
2. After typing the note, you realize that you got a few details wrong. Please make these edits:
- In the first paragraph, change Friday to Thursday.
- Change 5:30 to 5:00.
- Move the entire sentence Jenny arranged for Jeff's roommate to keep him away until 7:00. to the end of the second paragraph, after no later than 6:45.
When you are done, please save the file. You can use the same filename.
3. Actually, Jeff prefers to go by Geoff, and Jenny prefers Ginny. Please replace every occurrence of Jeff with Goeff, and all instances of Jenny with Ginny. When you are done, please save the file. You can use the same filename.
4. You'd like to make a copy of the note, using a different name that you can find more easily later. Please save a copy of this note as Geoff surprise party.txt in the Documents folder. For the purposes of this exercise, you do not need to delete the original file.
5. You decide the text in the editor is difficult to read, and you would prefer to use a different style of text. Please change the text to DejaVu Sans Mono, 12 point.
6. You decide the black-on-white text is too bright for your eyes, and you would prefer to use different colors. Please change the colors to a dark color scheme.
A file manager
When we performed the usability test of Gedit, we also worked on a usability test for another GNOME application: the file manager. The GNOME team wanted to know if users could use the Search feature effectively to locate lost files, among other features:
1. Yesterday, you re-organized your files and you don’t remember where you saved the copy of one of the articles you were working on. Please search for a file named The Hobbit.
2. Files and folders are usually displayed as icons, but you can display them in other ways too. Change how the file manager displays files and folders, to show them as a list.
3. You don’t have your glasses with you, so you aren’t able to read the names of the files and folders very well. Please make the text bigger, so it is easier to read.
A "Settings" app
In 2016, the GNOME Design team reworked the Settings app. The previous design was a grid of icons that opened up dialog boxes, while the new design was a list of categories which opened a new pane with specific actions. We didn't have a working version of the new design, so we crafted a paper prototype test and used 23 scenario tasks that prompted testers to tap on the mockup which item they would try first to complete the task:
1. You set an alarm on your laptop. Now you want to make sure that it will be loud enough to hear from across the room. Where would you look in the settings app to change the volume of the alarm?
2. You want to simultaneously be able to view your web browser and a text editor where you are working on a document. You have two monitors and want to spread your work space across them. Which setting would you use to enable this function?
3. You want to have your Thunderbird account to be the email that opens whenever GNOME is asked to open an email client. Which setting would you try to make Thunderbird your official email?
4. You would like to know which version of GNOME you are running so that you can see if you are up to date. Which setting would you try to find out this information?
5. You are about to watch a movie on your computer. The internal sound is too low so you get out a Bluetooth speaker. Where in the settings app would you go to connect the speaker to your computer?
6. You are working on a few design projects and want to adjust the way your monitor handles colors so that none of the image quality is lost. Which setting would you try first to change the default colors?
7. You would like to be able to access photos from GNOME shell search. Which setting would you try first to see if photos can be indexed in a search?
8. You are a very skilled typist. You notice a lag on your keyboard and want to speed up the computers’ response to your keystrokes. Where would you look in the settings app for a way to change the keyboard speed?
9. You are going out of town for a week and having a friend house-sit for you. She wants to use your desktop computer while you are gone and you are happy to oblige. Your friend’s native language is Spanish while yours is English. You think it would be nice to set your computer to Spanish for her so that she can use it more efficiently. Where would you look to switch the language to Spanish?
10. You prefer watching movies with the video player VLC. You would like to set VLC as the video player that GNOME connects to automatically. Which setting would you try first to make the change?
11. In a few minutes you are expected to be in a two hour long lecture. You want to take notes on your laptop during the talk. Where in settings would you look to see how much battery charge you have left in order to know whether you have to run to your car and get the charger before the lecture starts?
12. You’re at the office and you want to show a work presentation with a projector. Which Setting would you try first to connect your laptop to the projector?
13. You’re preparing for a remote work meeting. You wish to set a password that you will give to your coworkers so they can see and control your screen during the meeting. With which setting would you first attempt to do this task?
14. You find that you don’t like only having 5 minutes of idleness before your screen locks on your system. Where in the settings would you go first to adjust the wait time from 5 to 10 minutes.
15. You just bought a new printer. Which setting would you choose first to connect the new printer to your computer over WiFi?
16. You are living abroad for several months in a country with a time zone 6 hours ahead of yours. Where in Settings would you go to adjust the time zone to reflect your new location?
17. You are listening to music on your computer. At the same time, you are torrenting a number of files. You don’t want the notification sound to disrupt your music each time a file successfully downloads. Which setting would you try first to mute the volume of the notification?
18. You don’t want to have alert messages come up when your screen is locked. Which setting would you try first to turn off these messages?
19. You are tired of looking at the default lock screen image on your computer. Where would you look first in the settings application in order to replace the lock screen image with a picture of your family that you have saved on your computer?
20. You want to be able to scroll with two fingers on your touchpad. Which setting would you try first to enable this scrolling on your touchpad?
21. You have the WiFi password for your favorite local coffee shop saved on your computer. While you are there one day working on your laptop the person next to you asks if you know the WiFi password. Where in settings would you look to locate the saved password in order to share it?
22. You’re at the office trying to get work done. The WiFi has gone down three times already today and you are frustrated with waiting. You have an Ethernet cable plugged in and set up already, which panel would you try first in the Network settings to switch over from WiFi to Ethernet?
23. You’re not getting a great signal to the internet. Where in the Network settings app would you go to change which WiFi network you are connected to?
The last three questions focused on the WiFi panel.
These scenario tasks allowed us to examine these core parts of the new Settings app:
- Settings: Can testers quickly navigate to the correct system settings, and change the settings to a new appropriate value?
- Sound: Can testers adjust volume settings for media, alerts, and other sound events?
- Displays: Adjusting display modes for mirroring between two screens, joining two screens, or displaying to a projector.
- Printers: Setting up a new printer, or adjusting settings for an existing printer. One possible use scenario would be changing the driver to get better printed output.
- WiFi: Configuration for new wireless networks.
A word processor
Several years ago, I worked with a group of developers to examine the usability of LibreOffice Writer as a desktop word processor. LibreOffice is a very mature word processor, but had developed a mythos that it was hard to use. So we started by writing personas to focus on a particular audience (undergraduate students) and crafted a set of scenario tasks to exercise LibreOffice Writer for writing class papers:
1. You’re writing a lab report for your Introduction to Physics class, but you need to change it to meet your professor's formatting requirements. Change your text to use Times New Roman 12 pt. and center your title
2. There is a requirement of double spaced lines in MLA. The paper defaults to single spaced and needs to be adjusted. Change paper to double spaced.
3. After going through the paragraphs, you would like to add your drawn image at the top of your paper. Add the image stored at velocitydiagram.jpg to the top of the paper.
4. Proper header in the Document. Name, class, and date are needed to receive a grade for the week.
5. You've just finished a physics lab and have all of your data written out in a table in your notebook. The data measures the final velocity of a car going down a 1 meter ramp at 5, 10, 15, 20, and 25 degrees. Your professor wants your lab report to consist of a table of this data rather than hand-written notes. There’s a note in the document that says where to add the table. [task also provided a 2×5 table of sample lab data]
6. You are reviewing your paper one last time before turning it into your professor. You notice some spelling errors which should not be in a professional paper. Correct the multiple spelling errors.
7. You want to save your notes so that you can look back on them when studying for the upcoming test. Save the document.
8. The report is all done! It is time to turn it in. However, the professor won’t accept Word documents and requires a PDF. Export the document as a PDF.
It's an art, not a science
I wanted to share these scenario tasks to demonstrate how we wrote scenario tasks that set a brief context then asked the tester to do something specific. The key in writing effective scenario tasks is that the task should provide a clear goal, such that upon reaching the end, the tester should think "Yes, this is it." And the result should also be clear enough that the moderator will recognize that the tester has reached the correct endpoint.
Writing scenario tasks can be more of an art than a science. The key to writing good scenario tasks is to avoid using keywords from the design. Writing the right scenario task might require several revisions and rewrites. But the payoff to having well-constructed scenario tasks is that you'll get useful feedback from your usability test.
Jim Hall is an open source software advocate and technical writer. At work, Jim is CEO of Hallmentum, an IT executive consulting company that provides hands-on IT Leadership training, workshops, and coaching.