Scientists studying LGBT youth may have a big problem on their hands: meddling kids. More specifically, meddling kids who think it’s funny to identify as gay or lesbian on surveys—and who then give exaggerated responses to other questions about, say, substance use or eating habits.
That phenomenon, in turn, could be inflating certain estimated health disparities between LGBT youth and their peers.
NYU economics professor Joseph R. Cimpian, whose new study in the American Journal of Public Health explores this emerging problem, said that he was tipped off to its potential scale when he noticed how frequently the ostensibly gay kids on one survey reported being blind—way beyond what would be expected.
ADVERTISEMENT
“What we found is that ‘gay’ kids are way more likely to be blind and to be deaf and to have three or more children of their own and all sorts of things,” he told The Daily Beast. “When you look at these data, you think, ‘This is ridiculous!’”
There was only one conclusion: “Clearly the kids are messing with us.”
For their new study, Cimpian and his team looked at data from the CDC’s 2015 Youth Risk Behavior Survey, which is administered to high-schoolers, looking only at jurisdictions where the YRBS asked about sexual orientation.
Then, through a sophisticated statistical analysis that required immense computing power and machine-learning techniques, the researchers were able to find many of the so-called “mischievous responders” in the data set.
The simplest way to describe their method is that they identified—and then weeded out—youth who said they were “gay” but who also said that they were preternaturally tall and ate carrots four or more times a day every single day. That’s the sort of mischief that requires you to “get into the mindset of who that type of kid might be,” says Cimpian, and imagine what they find humorous. Carrots seem to be a favorite.
“That’s a pretty tempting response option, we find, for the mischievous responders,” Cimpian says.
And although Cimpian readily admits that NYU professors “may not necessarily be the best at figuring out what kids are going to think is funny,” the data offers strong clues.
“We do know that we can get very different responses particularly for high-risk, low-frequency kind of outcomes,” he said. “Things like excessive drug use and excessive alcohol use.”
When there are a large number of “mischievous responders,” all those lies about drug and alcohol use can add up. As Cimpian reports in the study, removing the “mischievous responders” had a significant impact on “overall estimates of LGBQ-heterosexual youth disparities, especially among male youths” but only in certain areas. (The 2015 YRBS did not ask about transgender status, hence the absence of the “T” in the acronym.)
“Drug- and alcohol-use disparities were among those most affected by suspect data, whereas disparity estimates for being bullied, feeling sad or hopeless, and thinking about suicide were not noticeably affected by suspect cases,” the study notes.
What we learn from that finding is that “mischievous responders” think some responses are funny but not others. For example, a 12-year-old saying that he has used heroin over 40 times has a certain panache to it compared to, say, disclosing suicidal ideation. But that could have the net effect of making it seem like gay and bisexual boys are abusing substances at much higher rates than they actually do.
“I think that [my study] suggests that the disparities are, particularly among males, not as big as the literature previously would have suggested,” Cimpian told The Daily Beast. “And some of those disparities actually diminish to basically nothing. They’re definitely not statistically significant in many cases—but, not only that, the actual differences are virtually nothing.”
For example, differences in cocaine and ecstasy use fell dramatically as the “mischievous responders” were sifted out of the YRBS data. Even by removing the 10 percent of responders who "provided the most unusual patterns of responses," disparities between straight and gay or bisexual boys decreased by an average of 28 percent, the study notes.
Changes among the girls, however, were nowhere near as large—“mischievous responding isn’t as prevalent of a phenomenon among females,” says Cimpian—and the disparities around things like suicidal ideation and bullying remained significant across both genders.
In that sense, Cimpian’s study underscores the statistical strength of the LGBT mental health disparity among youth. Even when you remove the meddling kids, it still stands.
“It actually shows that that’s a very robust finding,” Cimpian explained.
Still, the impact of mischievous responders on the substance use data is a potential problem, especially considering that the CDC uses the data to formulate public policy. Unfortunately, Cimpian says, mischievous responding is hard to identify—and perhaps even more challenging to prevent.
Cimpian told The Daily Beast that he’s currently running follow-up analyses on a supercomputer. That’s the level of technical power required to go through a large data set and filter out the effects of probable mischievous responding. Doing that by hand—or even on less sophisticated technology—would be incredibly laborious.
“Now we have the technology to comb through data in ways that we as humans wouldn’t want to go through,” says Cimpian.
Given how annoying it is to weed out mischievous responding after the fact, one might think that researchers would want to find ways to stop it before it happens. They do. But techniques have not been studied. And mischief, it seems, is infamously irrepressible.
“We don’t know how to discourage it,” Cimpian admits. “Mischievous responders are kind of difficult to study. When you try to study something, you actually alter it by measuring it, so if somebody knows that they’re going to be discovered as a mischievous responder, they may not actually reveal their mischievous responding tendencies.”
Even changing surveys to limit mischievous responding may only mask the problem.
Several years ago, for example, colleagues of Cimpian’s made the mistake of having write-in options for height and weight, which had an obvious result: “Some kids were saying that they were 10,000 feet tall or that they weighed 666 pounds.”
So, they made it a drop-down menu instead—but then the “mischievous responders” just selected the highest and lowest possible heights, like 7’11” and 4’0”. That means the “mischievous responders” will just get lumped in with the kids who are actually at the extremes.
Trying to stop mischievous responding, then, might be more challenging than using a supercomputer.
The allure of depicting yourself as a 10,000-foot-tall, carrot-chomping, gay cocaine addict may, for some youth, be too great to resist. If Cimpian’s being completely honest, some of the grown-ups who have looked at this problem might have done the same thing when they were younger.
“A lot of times when I even talk to fellow faculty members about this, they say things like, ‘Oh, I would have been the kid that you would call a mischievous responder,’” says Cimpian.
