ChatGPT is an okay technical writer
ChatGPT can help streamline work by automating more mundane writing assignments. However, approach carefully.
You don't have to look very hard to find an article expounding on the benefits of ChatGPT. But how well does ChatGPT really do? I put ChatGPT to the test to examine how it could tackle certain technical writer tasks.
How generative AI works
Chatbots like ChatGPT are called "generative" text processors. They are software that has been "trained" on a large body of work - such as articles from the Internet and other material - until the AI has developed a strong relationship model of the text database.
At a high level, ChatGPT and other generative AI responds to a prompt by first dissecting the user's input to determine the subject, the context, the requested tone, and other parameters. As ChatGPT produces output in response to the prompt, the AI basically works through a statistical model of what word should come next in the context of the words it has already produced.
This description simplifies the process perhaps too much, but it is enough to highlight that these generative AI don't actually understand the text they produce. Instead, it's closer to a mathematical model.
With this simple model, you might see that the more training the AI has received on a certain topic, the more text it can generate on that topic. The generative text will be a hybrid of other work the AI was trained on. For topics with lots of background information, the AI is more likely to generate correct responses - with the limitation that the training material should be correct.
Writing how-tos
As an experiment, I tasked ChatGPT to generate several kinds of articles that a technical writer might produce. When I'm not writing for Technically We Write, I often write about Linux, open source, and programming for websites, journals, and magazines. I started by asking ChatGPT to write a how-to on a topic that I might also write about:
>> Write a brief how-to on how to use the printf() function in C
This is the full response:
The printf() function is a standard C library function that is used to print output to the screen. It is part of the stdio.h header file, so you will need to include this header at the top of your program in order to use the printf() function.
Here is a brief overview of how to use the printf() function:
Include the stdio.h header file at the top of your program:
#include <stdio.h>Use the printf() function to print output to the screen. The general syntax for the printf() function is:
printf("format string", argument1, argument2, ...);The format string is a string that specifies how the output should be formatted. It may contain placeholders for the arguments that you want to print, which are represented by % followed by a specific format specifier (e.g., %d for integers, %f for floating-point numbers, %s for strings, etc.).
Specify the arguments that you want to print in the order that they appear in the format string. The number and type of arguments should match the number and type of placeholders in the format string.
For example, to print an integer and a string on separate lines, you might use the following code:
#include <stdio.h>
int main()
{
int x = 42;
char* str = "Hello, world!";
printf("%d\n", x);
printf("%s\n", str);
return 0;
}This would output the following:
42
Hello, world!You can find more information about the printf() function and the available format specifiers in the C standard library documentation.
I chose printf because it is a common way to print output from a C program. Much has already been written about C programming, and I think it's safe to assume every reference to C programming includes a description of using printf. From that assumption, it's also likely that ChatGPT has been well trained on this programming concept.
How did ChatGPT do
This is a brief but accurate description of how to use printf in a C program. The name printf means to print formatted output, and the function uses a format string with format specifiers as "placeholders" for different values like integers, floating point values, characters, and strings.
ChatGPT does a good job here. The generated text doesn't appear to exhibit bias, the sample code is correct, and the output matches the code provided. Overall, this is a fair technical writing sample.
However, it is very short. The prompt asked for a "brief how-to" so it should not be too surprising that ChatGPT provided a short response. But ChatGPT also has a character limit for each reply, which is another limitation to using it for long works.
Other samples
I asked ChatGPT about other technical writing topics, all related to Linux and programming. Here are a few examples that I asked:
>> Describe how to use the puts function in the C programming language
The puts function is another method to produce printed output from C programs. The name represents that it will put a string on the standard output device, usually the user's screen. Like the earlier printf example, ChatGPT generated a correct response to the prompt. I did not identify any issues, errors, or biases.
>> Write a C program to add the numbers from 1 to 10
ChatGPT responded by writing a mini-article about how to write a program to sum a list of numbers. ChatGPT started by listing the program source code, then described how the code worked.
The program was syntactically and idiomatically correct; the program, while a simple example, is one that a programmer might use to teach someone else about programming.
>> How do I write a web page in HTML?
ChatGPT expanded my rather plain prompt and generated a mini-article about writing HTML. It opened with an example HTML page, then explained how the different parts fit together.
The example was rather terse, however. It didn't explain how tags worked in an HTML document, or provide any hints that tags don't need to be on separate lines.
ChatGPT as a technical writer
I asked ChatGPT about more than a dozen prompts that I might write about in technical writing. The responses were consistently "good" - not outstanding, but "okay." For technical writing topics like Linux and programming, ChatGPT did a fair job.
There's a broader question here about the ethics of using ChatGPT in technical writing. I'll address this briefly with my view that if you claim authorship, you should be the sole author of the piece. If you use ChatGPT to write material for you, I think the obvious ethical line is that you should not claim it as yours; don't put your name on it if you didn't write it. That's the approach I took when I "wrote" a book with ChatGPT; I was only the editor.
But if your task is to write technical how-to articles, ChatGPT is available to you as a writing tool. The responses for this narrow range of popular technical topics were satisfactory. However, approach ChatGPT carefully. Looking through the responses from ChatGPT in this exercise, I identified three key takeaways:
- Verify everything ChatGPT generates for you. For popular topics, it is likely to be correct. However, ChatGPT will write nonsense with equal confidence. It is your responsibility to fact-check everything ChatGPT produces.
- Watch for small errors. ChatGPT sometimes produced minor errors in its responses. For example, when I asked ChatGPT to cite a book in APA format, it generated a response that was correct - except it indicated the editor was the author. In this case, the correct APA citation would have been a collection of essays from multiple authors, with a single editor. Instead, ChatGPT claimed the editor as the "author" of the book.
- Be mindful of omissions. Even when ChatGPT generates correct responses, it may not include all of the necessary information. The details it provides are correct, it's just missing other information. This would be like asking someone for a cake recipe and their reply didn't include sugar. Yes, you still need to mix the flour and the eggs, and bake the cake at 325 for 30 minutes. But the result won't be a birthday cake.
Jim Hall is an open source software advocate and technical writer. At work, Jim is CEO of Hallmentum, an IT executive consulting company that provides hands-on IT Leadership training, workshops, and coaching.