Much has been made about ChatGPT replacing the jobs of journalists, writers, and even musicians—and for good reason. The large language model (LLM) has the uncanny ability to simulate human writing in a way that we hadn’t seen before in a chatbot. It’s made large swaths of professionals confront a fear that they wouldn’t have dreamt of happening in a million years: Their jobs could be automated.
Some new research published in a paper Thursday in the journal Science adds another notch to that column, and is sure to elicit mixed feelings for many. A team of researchers at MIT found evidence that suggests that LLMs like ChatGPT and Bard can not only enhance the productivity of professional writers, but it could even render certain types of writing jobs obsolete.
“A potent generative writing tool such as ChatGPT could conceivably either displace or augment human labor,” the study’s authors wrote. “ChatGPT could entirely replace certain kinds of writers, such as grant writers or marketers, by letting companies directly automate the creation of grant applications and press releases with minimal human oversight.”
ADVERTISEMENT
However, the authors did add that it also had the potential to merely augment the jobs of writers. This would lead to an increase in productivity and quality of their work—especially if the worker was lower skilled in those roles.
The team recruited 453 college-educated professionals and tasked them with completing occupation-specific writing tasks. The jobs that they were asked to emulate were “marketers, grant writers, consultants, data analysts, human resource professionals, and managers.” They were then asked to write two assignments like press releases, emails, and reports in a 20- to 30-minute window. The participants were also incentivized with a base payment of $10 with up to $14 in bonus pay if their work was high quality.
Half of this group were allowed to use ChatGPT to complete their second assignment using the LLM, while the other half were told to sign up for a commonly used non-LLM writing tool called Overleaf. Eighty percent of the ChatGPT group opted to use it while less than 5 percent of the control group used the writing tool.
The results were stark. The time it took for the ChatGPT group dropped by an average of 11 minutes—that’s a 40 percent decrease from the time it took them to complete their tasks before. The control group took an average of 27 minutes to complete their task while the other group took just 17 minutes.
The writing was also graded by evaluators who found an 18 percent increase in quality of the assignments from the ChatGPT group when compared to the control. The data also showed that those who scored poorly in the first task performed better with the help of the chatbot. In fact, their quality increased by nearly 50 percent with the introduction of ChatGPT.
“College-educated professionals performing mid level professional writing tasks substantially increased their productivity when given access to ChatGPT,” the study’s authors wrote. “The generative writing tool increased the output quality of low ability workers and reduced time spent on tasks for workers of all ability levels.”
The authors note that there were several limitations including the fact that the writing ChatGPT was asked to create didn’t require a lot of context and factual accuracy, and it could be fairly generic. That means that writing jobs such as screenwriting, song writing, and journalism (whew) might be less impacted by these findings in the future… right?
Well, not quite.
The authors did note that certain roles in advertising and communications could be impacted depending on society’s acceptance or rejection of ChatGPT being used to sell them products. They added that it could entail “a reduction of employment in those sectors as fewer workers are needed to meet the same static demand.”
Perhaps most disconcertingly, though, the authors said that LLMs have the potential to “monitor or evaluate workers and avoid paying higher wages.” This is something we’ve already seen play out in limited to extreme extents in certain cases. The Hollywood Writers Strike is a prime example, as is the recent spate of media companies such as Gizmodo and Buzzfeed utilizing AI to churn out low quality, inaccurate, and shoddy content.
So it might not necessarily matter whether or not it boosts quality for individual workers or not. If the owners of these companies think that they can save and make more money by replacing their employees with LLMs like ChatGPT, they will do so. Sure, it might not last as people start to seek out actual high-quality human-created content. However, it’s still a rush to the bottom before we start to make our way back up.
