Did you hear that “Psychologists have identified four different types of drunk people”? So we are told by The Daily Telegraph, discussing a recent research paper by Rachel Winograd, Douglas Steinley, and Kenneth Sher that used statistical methods to characterize student drinkers as “Ernest Hemingway,” “Mary Poppins,” “Mr. Hyde,” and “Nutty Professor.”
Our diversity of press outlets gives us the opportunity to see this story from many perspectives.
If you want the credulous take, check out The Atlantic article, with its sober subhead, “When people act differently under the influence, it could be a sign of an alcohol problem”:
For a study recently published in Addiction Research & Theory, researchers questioned 187 pairs of “drinking buddies”—undergrads who frequently drank together and knew what their friend was like when intoxicated. Participants were asked how much they drank and how often, as well as whether they ever experienced any negative consequences from drinking, such as lower grades, regrettable sex, or craving a drink first thing in the morning. They were also asked to describe what they’re like when they are drunk by using the “big five” personality traits, which are openness, conscientiousness, extraversion, agreeableness, and neuroticism. Their buddies then corroborated (or contradicted) these personality assessments. The researchers grouped the respondents into four clusters…”
This is a fair and accurate description of the study, with the only problem being a seemingly desperate desire to believe. Check this out, from later on in that Atlantic article:
The rigor of these categories leaves something to be desired—they were somewhat arbitrarily constructed based on the recalled observations of a relatively small number of college students. And the clusters obviously don’t capture all the possible permutations of drunk behavior. (Let’s hear it for the sleepy drunks who effusively make impossible-to-keep brunch plans.)
But it’s significant that the people who change the most and are the least fun to be around when drunk are also the most likely to have alcohol-related issues, like blackouts or fights. That suggests “drunk personalities” could be clues to the trajectory and nature of drinking problems… .
More studies need to be performed before that can happen, but this one is a start…
The reporter identified the statistical problem and then just set it aside. The problem, though, is that such clusters are arbitrary. These sorts of data can be divided into any number of clusters, and then it just becomes a contest for who comes up with the catchiest names, such as Mary Poppins etc.
A more jocular approach comes from Time magazine:
Psychology researchers from the University of Missouri at Columbia have published a study in Addiction Research & Theory attempting to bring the conventional wisdom that there are many distinct ways to be drunk to its logical, scientifically-based conclusion… .
The study authors hope to use these categories to tailor future alcoholism interventions to particular personality types. Meanwhile, you can use them to take bets on how many beers in your Nutty Professor friend will have had enough to start flirting with that brunette by the jukebox.
This is a saner take on the study, but it, too, bothers us. Even setting aside any concerns we might have regarding joking about mental illness, we are concerned about the effects of this sort of headline-grabbing study on the general reputation of science. “Science to the rescue,” indeed.
As statisticians, how should we think about this sort of study? From long experience we know that the number of clusters found by a cluster analysis is not a scientifically meaningful quantity. A set of 374 drinkers can be categorized as falling into one cluster, or 374 clusters, or anything in between. The number 4 was obtained by following a particular statistical procedure, but these sorts of cluster-analysis methods are all about prediction, and they depend crucially on the amount of data gathered on each person. The more data, the more that people can be differentiated. Is it useful to arbitrarily categorize drunks in this way, or is it counterproductive?
On the other hand, assuming the data were gathered carefully, the analysis can have value. What is needed is to validate the clusters by making predictions on new data, to see if it is really the case that, as claimed by the authors, “These results support use of the five-factor model to characterize clinically meaningful subgroups of sober-to-drunk differences in trait expression.” Until we see such validation, no, we don’t see these results offering support for any such theory, given that any data can be cluster-analyzed.
We don’t want to pick on the authors of this particular study, which seems completely reasonable as an exploratory data analysis and worth publication on those grounds. And we appreciate their recommendation of the limitations of the study.
To us, the challenges here involve not statistics but publication. Is there a way in which a scientific study can be published as being frankly exploratory without purporting to make general claims about the world? And is there a way in which the news media can report speculations and exploratory findings without the hype?
