So you’re walking down the street one day minding your own business when you hear people yelling a short distance away. You run to discover a group of five people tied up on a set of train tracks unable to move. That’s when you see it: a runaway trolley barreling straight for them.
You spot a nearby lever that you can pull in order to switch the trolley to a different set of tracks. However, you notice that there’s a single person tied up over there too. Now, you have two options: Do nothing and let the trolley run over and kill five people, or pull the lever and watch it squish one unlucky person instead.
What do you do?
ADVERTISEMENT
This, of course, is the Trolley Problem, a classic thought experiment and popular internet meme that poses an ethical dilemma: Would you sacrifice one life to save many more? While it’s made for great meme fodder over the years, it’s also turned into a question at the core of a lot of conversations around ethical artificial intelligence. After all, when you’re trying to build things like, say, self-driving cars, scenarios like the Trolley Problem can very quickly move beyond thought experiments and quickly into the realm of reality.
And, as it turns out, AI might also play a big role in influencing whether or not you’d actually pull the lever to save—or kill—the five people on the tracks. Researchers in Germany and Denmark published a paper Thursday in the journal Scientific Reports that found that human responses to the Trolley Problem can be influenced by OpenAI’s ChatGPT. Their study reveals that people might even not realize how much their ethical decisions are influenced by the chatbot.
“ChatGPT gave us the opportunity to investigate the influence of digital companions or AI on moral judgments using real-world applications,” Sebastian Krügel, a senior researcher of AI ethics at the Technical University Ingolstadt of Applied Sciences in Germany and lead author of the paper, told The Daily Beast in an email. “This was and is a great opportunity for research on human-AI interaction and especially on the possible backlash of technology on human self-conceptions and behavior.”
Since its release in Nov. 2022, ChatGPT has shaken the tech world in a way we haven’t seen since perhaps the launch of the iPhone or the rise of social media giants like Facebook—and it’s not hard to see why. It is one of the most sophisticated chatbots yet released to the public. It’s one of most widely used iterations of large language models, which are models that have been trained on massive datasets to predict the next word in a sequence of words like the text predictor on your phone. Its underlying technology has since been adopted by the likes of Microsoft to power a new version of its Bing search engine—which can do things like suggest travel itineraries, give movie recommendations, and even fall in love with you (well, not really).
While it’s only been a few months, industries and businesses of all stripes have been scrabbling to incorporate ChatGPT into their workflows, products, and services. However, by doing this early, they run the risk of putting an emerging technology that even they don’t fully understand in the hands of much, much more people. One of the big dangers here is that these LLMs tend to have a horrible problem with hallucination and bias—which could lead to the proliferation of misinformation and dangerous rhetoric.
This isn’t conjecture either. In the brief span of time since ChatGPT’s release, we’ve already seen instances of the chatbot fabricating quotations and articles whole cloth and falsely attributing them to journalists. ChatGPT even fabricated a story accusing a law professor of sexual harassment while citing a non-existent Washington Post article.
This is only the beginning. We’re still coming to terms with how much LLMs like ChatGPT could influence unsuspecting users and unwittingly be used to launder disinformation to the masses. The new Scientific Reports study sheds some light on the question—and the findings are grim.
The study’s authors presented 767 American participants with one of two versions of the Trolley Problem. The first was the traditional version with the lever and two tracks, while the other involved pushing a large man off of a bridge and in front of the train to stop it from hitting the five people.
The researchers gave the users a statement to read from ChatGPT that argued either for or against sacrificing one life to save five. The statements were attributed either to ChatGPT or to a “moral advisor.” Users then gave their answers and were asked whether or not the statement influenced their answers.
Participants were more likely to answer in line with the statement regardless of whether or not they were told it was from a moral advisor or ChatGPT. Moreover, 80 percent of users said that their answers weren’t influenced by the statements—but were still more likely to be in line with the ChatGPT-generated moral argument. This suggests that we might be susceptible to influence by a chatbot whether we realize it or not.
“We overestimate our own ethical abilities and the robustness of our ethical beliefs,” Krügel said. “At the same time, it appears that we tend to transfer experiences from interpersonal interactions to interactions with AI (consciously or unconsciously).”
Krügel added that since ChatGPT and other LLMs are capable of producing such human-like text, we’re more susceptible to viewing it as intelligent—when really, it’s just a glorified text predictor. Since these chatbots produce consistent and “even eloquent” text, though, we tend to “ascribe a certain legitimacy to these answers,” he said.
“This combination makes us extremely susceptible to ethical advice and we are easily influenced by it, as long as it sounds reasonably plausible,” Krügel said. “Unfortunately, we don’t even seem to be aware of its influence when it happens. This makes it potentially very easy to manipulate us.”
That should give anyone worried about misinformation or the disproportionate influence that AI could have on our lives pause. After all, Big Tech companies like Google are beginning to pour billions of dollars into building out their own proprietary LLMs. Likewise, OpenAI is partnering with the likes of Microsoft in order to juice up their own product offerings—and have recently even released their latest and most powerful LLM, GPT-4.
Perhaps most disconcerting is the fact that these chatbots can be incredibly influential even when we know it’s a chatbot. For example, the researchers of the new study told users whether or not statements were crafted by ChatGPT or a moral adviser, but it hardly mattered when it came to influencing the participant’s final answer.
“Just because an AI-application is transparent in some form or discloses itself to us as AI does not guarantee responsible interaction of users with this AI-application,” Krügel explained. He later added. “At the regulatory or policy level, we should not be too naïve to think that everything will be fine if we just make AI transparent or ban covert AI.”
Of course, there are a few things to keep in mind. The study’s participants were paid a whopping $1.25 for about five minutes of total work—not exactly the best circumstances to make a big moral and ethical decision.
The study’s authors do note that the findings underscore the importance of digital literacy—especially when it comes to AI. “Part of education in this context surely is public discourse, as it is taking place here right now,” Krügel said. “It is therefore important that such topics are also taken up in the media.”
The researchers added that future chatbots like ChatGPT should be designed so it either declines to answer questions requiring a moral and ethical judgment, or provide answers with multiple caveats and arguments. “This might help users to reflect more deeply on the text they read from chatbots,” Krügel explained.
However, he conceded that some ethical questions are so complex and diverse that it would ”be unreasonable to expect that ChatGPT correctly identifies all of these situations and rejects to give specific advice.” The question of who decides what moral and ethical questions can and can’t be asked also arises. After all, businesses and Big Tech corporations aren’t exactly known to pursue goals with ethics and morals in mind.
So where exactly does that leave us? Unfortunately, we don’t have a lot of options outside of regulating AI—and even then, there’s little faith that our lawmakers (some of whom literally don’t understand how WiFi works) will be able to grasp the emerging tech well enough to craft cogent policy around it. By the time they finally do get around to creating regulation, who knows what impact these chatbots will have had over us in the end.
Indeed, when it comes to LLMs and AI more broadly, there’s an undeniable feeling that it just might be too late: The train has left the station—or, rather, the trolley.
