stories-matter My background in usability testing

November 13, 2025, is World Usability Day.

I've explored a lot of interests over my career. While my primary career trajectory has been in IT and IT leadership, moving from a systems administrator (my first job) to IT management and IT leadership (including CIO in higher education and government), I've also maintained a strong interest in other areas too. Two of these other areas include open source software and technical writing.

My work in open source software goes back quite a ways, but I'll summarize by highlighting that I started contributing to open source projects when I was an undergraduate university student. I modified GNU Emacs to run on the Apollo DomainOS system, I ported a small C compiler to work on HP-UX, and I started an open source version of the DOS operating system.

Much later, I was encouraged by a friend to write articles about open source software, and I found that I enjoyed writing articles about how to get started in open source. I shared my articles in places like Linux Journal, Linux Magazine, Opensource.com.

My first usability test

Around 2012, I entered a Master's program. The MS in Scientific and Technical Communication program matched my interests very well: It had a strong connection to my role as a CIO in higher education, where I needed to write strategy documents, positioning statements, and other written content to communicate with the campus. And the program connected to what I already enjoyed doing—writing about open source software.

During that program, I had an opportunity to explore usability testing. Usability is an important part of technical communication. By learning about usability, you actually build a very strong understanding of audience and how people approach content. I was fortunate to do an independent study about usability testing. This was a deep dive that started with "what is usability testing" and "how to test usability," so I could learn more about the different kinds of usability tests and why you would do each kind of test.

To fully explore usability testing, I designed and executed a usability test. My advisor encouraged me to do this test on a topic I cared about, so I would be more invested in doing well and learning from the results. I contacted the GNOME Project, an open source desktop environment for Linux, and connected with the GNOME Design team.

I had used GNOME since version 1.x (late 1990s); by then, GNOME was at version 3.x. Starting with version 3 (2011), GNOME completely changed the interface. This upset a lot of people, who called it "difficult to use." I wanted to do a usability test with real users to examine the usability of GNOME 3, and suggest improvements.

That usability study was my first real usability test—and as such, it was stressful to do, but it was also a lot of fun. In the end, I identified several areas of GNOME that could be improved, and the GNOME Design team actually used those suggestions to improve the next version of GNOME.

My next big usability test

At the end of the MS program, students have to complete a capstone project. This is similar to a thesis project, but much more focused; perhaps a better parallel is a practicum. We could choose our own topic; I decided to write about usability testing.

I contacted the GNOME Design team and coordinated with them to do another usability test. I did a much deeper analysis of the usability test, and provided a detailed examination of several usability issues. But as a capstone project, I also took it to the next level, and used the usability test to help draw similar conclusions about usability in general in open source software projects. In my paper about Usability Themes in Open Source Software, I shared several suggestions for open source software projects to improve their usability, including:

  1. Familiar design
  2. Consistent experience
  3. Menu clarity
  4. Obvious results
  5. Visual relationships

Open source usability

After earning my MS degree, I developed a strong interest in the usability of open source software.

I also mentored others in usability testing in open source software. For example, I volunteered with Outreachy, which provides paid internship opportunities for women and others who are underrepresented in open source software projects. With these internships, I was glad to work with GNOME on more usability tests, continuing to improve the GNOME desktop environment. (I was also glad to see that at least two of the interns I worked with have since found their own careers in usability testing.)

Outside of Outreachy, I also helped other open source projects with their usability, often responding to questions, giving advice, and (sometimes) mentoring those who were working to improve the usability of their app, website, or both. I love working with open source projects, and usability is one area that can really raise the visibility (and first impression) of open source.

Through it all, I also wrote articles and gave many conference talks about usability and how open source projects could improve their usability. You can find several of my articles about usability on various websites like Opensource.com (although Opensource.com stopped publishing in 2023).

Find your own path

I encourage anyone who is interested in usability testing to jump right in. You don't have to be an expert in usability testing to design and execute your own usability test. The basic process to running a usability test includes these steps:

  1. Understand the users. Write "personas" to describe the kinds of people who use the system.
  2. Understand what real people do. Write "scenarios" to identify how those people use the system.
  3. Define the test. Create a series of tasks that set a brief context then ask the tester to do something.
  4. Run the test. Ask at least 5 people to follow the tasks you wrote, and observe what they do and say. Take detailed notes.
  5. Analyze the results. After you've tested with 5 people, you can identify themes about where the system works well, and what areas could be improved.

November 13, 2025, is World Usability Day which is an excellent opportunity to learn more about usability and usability testing. If you want to explore usability on your own, I recommend starting with these excellent overviews and resources on Technically We Write:

Overviews:

Resources:

photo of jhall

Jim Hall is an open source software advocate and technical writer. At work, Jim is CEO of Hallmentum, an IT executive consulting company that provides hands-on IT Leadership training, workshops, and coaching.