Login
John Oliver Warned Against Cherry-Picked Studies. Then He Cherry-Picked His Own.
In 2016, the HBO comedian and political commentator John Oliver produced a terrific segment on how TV pundits routinely cite garbage science. After showing clips of newscasters and commentators making absurd claims such as, "A new study showing that drinking a glass of red wine is just as good as spending an hour at the gym," and The Today Show's Al Roker commenting, "You find the study that sounds best to you, and you go with it," Oliver commented, "No! No, no, no, no, no, no, no! You don't just get to cherry-pick the parts that justify what you were going to do anyway."
Then Oliver added a useful rule: "If they are going to keep saying 'a study says,' they should have to provide sourcing and context or not mention it at all." (Though the segment specifically targeted the misuse of "scientific" studies, all the same principles apply to the social sciences.)
What's so noteworthy about that 2016 segment is that in his other monologues, on issues ranging from SWAT teams to homelessness, Oliver has routinely ignored the principles he laid out. In his overly simplistic takes on complex policy issues (often designed to dunk on his ideological opponents for comedic effect), he often fails to provide appropriate context and appears to be guilty of choosing research "that sounds best," as Roker put it, or at least that supports whatever point he's determined to make.
Oliver is one of America's most influential pundits, so he should be held to the highest of journalistic standards. And Last Week Tonight is indeed more diligent than many TV news programs. The show employs researchers who interview the authors of the studies it cites. Oliver is a comic, not an academic, so we can excuse him for not posting footnotes, and he routinely cites his sources on screen. When we emailed the show's producers to point out problems with their research, to their credit, they sent back a detailed response that we'll quote from in this article. (In a few cases, they pointed to Reason articles using some of the same studies, albeit with full citations and more context. In cases in which their criticisms were correct, we've annotated the original articles.)
Many of the show's researchers are former reporters and editors who abide by journalistic standards. Nevertheless, the show has a long track record of citing social science findings without the appropriate context, massaging them to support whatever point Oliver is trying to make.
Rent
In his excellent 2016 segment on scientific studies, Oliver noted: "There is a lot of bullshit currently masquerading as science. Not all scientific studies are equal. Some may appear in less-than-legitimate scientific journals, and others may be subtly biased because of scientists feeling pressured to come up with eye-catching positive results."
Cut to a 2022 segment titled "Rent," and we find Oliver citing a study that his wiser self would have characterized as "bullshit."
"Right now, there is not a single county in the U.S. where a worker earning minimum wage can afford a modest two-bedroom rental home," he said. He was citing a study designed to produce an "eye-catching" result. It was published not by a reputable journal, but by the National Low Income Housing Coalition, an advocacy group that lobbies for government spending on housing and rent subsidies, including expanding the National Housing Trust Fund and increasing federal appropriations to the Department of Housing and Urban Development.
The biases are not particularly subtle. Why would a minimum-wage worker need a two-bedroom home? In the study, "can afford" doesn't mean "have enough money to buy the cheapest available," as most English speakers would interpret it. The authors use it to mean renting a two-bedroom home at the 40th percentile price point of the local market (that is, a little cheaper than the median unit), using no more than 30 percent of their income, excluding government benefits, pensions, or other nonearned support.
Oliver also didn't mention who these minimum-wage workers are. In 2025, according to the Current Population Survey, a third of hourly workers paid the federal minimum or less lived in a parent's or grandparent's home, a fifth were married and living with a spouse, and only one in eight lived alone.
When we asked the show's producers why they failed to disclose that their source was an advocacy group seeking government funds, they pointed to the onscreen credit and responded: "We trust our viewers' intelligence enough to think they might be able to read it and conclude that the National Low Income Housing Coalition is interested in building low-income housing, nationwide, possibly even as part of a coalition."
I don't share their confidence that most viewers can read the onscreen source line and tell from the organization's name whether it's an advocacy group. There's a double standard here: When Oliver doesn't like what a study says, he's not content to just trust his viewers; he makes it explicit, as he did with the "ethanol lobbying groups" mentioned above.
Oliver's producers wrote us: "When we rely on studies from advocacy groups, we look at the underlying data and speak to researchers to better understand it; we discard many studies as a result."
This one belonged in the trash.
Sports Betting
In a 2025 segment, Oliver cited an eye-catching statistic about how sports betting was driving a spike in bankruptcies. "Unsurprisingly, the rise of online sports betting has been found to correlate with a rise in gambling harms, with one study showing that in states that allowed it, there was a 28 percent increase in bankruptcies, translating to about 100,000 extra bankruptcies a year."
I'll give Oliver credit for saying "correlate with" rather than "cause," but the rest of that sentence ("in states that allowed it, there was a 28 percent increase") implies direct causality.
The bigger problem is that the "100,000" figure is clearly wrong. When I contacted the study's authors, they told me they don't know where that figure came from, and it doesn't appear anywhere in the study. Last Week Tonight's producers told us they pulled it from a Business Insider article, but in the video, Oliver said it came from the study.
If Oliver's researchers had done the same calculation as the authors, multiplying the per-person effect by the at-risk population, they'd have found about 30,000 extra bankruptcies a year, not 100,000, and a 6 percent to 7.5 percent increase, not 28 percent.
Oliver also left out important context: He was citing an unpublished working paper. The peer-reviewed version appeared after the Last Week Tonight segment aired, and it included substantive changes. Working papers from credible researchers, like this one, are valuable, but Oliver should have made it clear he was citing a preliminary draft circulated to other researchers for comment, not a published study.
Homelessness
In 2021, Oliver did a segment making a case for a policy called "Housing First," which includes placing more homeless people in permanent supportive housing. The problem is that building and maintaining these units is expensive, so permanent supportive housing doesn't scale easily. Oliver set out to dismiss that objection by citing research showing the alternative is even more costly.
"One study in Florida," Oliver said, "found that, just between money spent on incarceration and emergency medical treatment, their communities and local governments had spent an average of just over $31,000 per person per year, when the estimated cost of providing them with permanent supportive housing would have only been around $10,000 per year."
The Florida study's finding was an outlier. The Central Florida Commission on Homelessness commissioned it, and Housing First advocates promoted it because it produced the headline they wanted. I give Oliver credit for noting that the study "tracked a decade's worth of spending on just 107 chronically homeless people," but he left out additional context.
In 2018, a committee convened by the National Academies of Sciences, Engineering, and Medicine evaluated permanent supportive housing. It concluded that there is "no substantial evidence that [it] contributes to improved health outcomes" and "insufficient evidence" that it's cost-effective.
Studies exist on both sides of this issue, and we can debate whether permanent supportive housing is more cost-effective than the alternative, but Oliver cited one study and implied it was conclusive.
Oliver also neglected to mention that the paper was written by a consulting firm led by a supportive-housing advocate, though if you looked closely, you could see the organization's name on the screen as he delivered his monologue.
Ethanol
Oliver and I agree that the federal government shouldn't be subsidizing ethanol. But in a 2024 segment titled "Corn," Oliver pointed to a study claiming that ethanol is more harmful to the environment than gasoline and implied that only self-interested lobbyists say otherwise.
"While ethanol lobbying groups have long argued that it slashes greenhouse emissions," he said, "one recent study found that corn ethanol produced under the renewable fuel standard has a carbon footprint at least 24 percent higher than regular gasoline."
What Oliver doesn't mention is that the 2022 study he cited was highly contested. It was challenged not only by "ethanol lobbying groups" but also by the U.S. Department of Agriculture under the Biden administration. The study's claim that ethanol has a higher carbon footprint than gasoline also contradicts a 2010 Environmental Protection Agency projection that ethanol would be about 20 percent cleaner than gasoline. Argonne National Laboratory, running the government's standard model, credits ethanol with cutting greenhouse emissions by more than 500 million tons in 15 years.
The problem isn't that Oliver is wrong; it's how he presented this study. Whether ethanol mandates are good or bad for the environment is a complicated, contested issue, with strong studies on both sides (and plenty of weak ones). Plucking one from the pile and reporting its findings is misleading.
Ethanol subsidies are bad because they involve politicians using mandates to buy farmers' votes, and top-down mandates generally lead to less optimal decision-making. But it's not clear that ethanol increases carbon emissions.
SWAT Raids
In a 2014 segment on police militarization after the fatal shooting of Michael Brown in Ferguson, Missouri, Oliver said, "There has been a massive rise in both the number and the use of police department SWAT teams," and showed a news clip that stated: "The number of SWAT raids has gone up by 1,400 percent since the 1980s. An estimated 50,000 now take place every year." Instead of mocking the clip, as he often does, Oliver cited it approvingly.
So where did the claim of a 1,400 percent increase come from? It's based on a voluntary mail-in survey, which isn't a reliable way to gather data. No official count of SWAT raids exists, and almost nobody tracks them, then or now, so the figure rests on departments' self-reporting of how often they "deployed," with no shared definition of what that means.
The 1980s figures that form the baseline for the 1,400 percent increase in SWAT raids come from a mid-1990s survey, making them especially unreliable. No one was counting SWAT raids at the time, and there's no way to verify them.
Oliver followed that clip by citing another study: "Just keep that clip in mind," he said, "when I tell you that 79 percent of SWAT deployments are now for executing search warrants, and most of those were for drug investigations."
Again, there's missing context. That 79 percent figure comes from a small, nonrandom sample generated by an American Civil Liberties Union records request; the organization could get data from just 20 of America's roughly 18,000 police agencies.
When we pointed out to Oliver's producers that the data on SWAT teams were stripped of their context, they replied that since this segment aired, "We've become more diligent about critically interrogating the sourcing on studies in our shows." They also argued that more "recent studies," if anything, suggest that the studies they cited "understated the true scope of things." And they pointed to instances of Reason citing the same data.
I agree with Oliver's position on SWAT raids, but he really should be agitating for better data. We're permitting state violence without anyone keeping the detailed records that could help us understand the costs and benefits. But that doesn't excuse citing two studies based on weak data and presenting them as fact.
Tipping
In a 2025 segment on tipping, Oliver said, "Recent analyses have found that poverty rates for tipped workers are over twice as high as for nontipped workers."
Twice as high as for which nontipped workers? The analysis he's citing included all nontipped workers in the U.S., including CEOs and neurosurgeons. Oliver, who reportedly makes $30 million a year, also pulled up the nontipped average. The right way to evaluate tipping is to match tipped and untipped workers by job: Waiters have about the same poverty rate as fast-food workers, and hotel maids about the same as other cleaners. If you control for job, age, and hours, the gap closes.
It's another study designed to generate a talking point, which isn't surprising since it was published by the Economic Policy Institute, a progressive, left-of-center think tank. The organization's name appeared onscreen as Oliver delivered his monologue, but, once again, viewers would need to hit pause to look it up and figure out what the Economic Policy Institute stands for.
Oliver's 2016 segment on "scientific studies" shows that he should know better. It's possible to construct arguments by citing good research responsibly, while providing the proper context and sourcing. It's possible to follow research where it leads, instead of retrofitting it to preconceived ideas. You just have to be willing to present data that capture reality in all its messiness, even if that means it won't generate an easy laugh line.
To keep up with our video series Wrong Number featuring Aaron Brown, and to receive bonus content and more, click here to enter your email address.
The post John Oliver Warned Against Cherry-Picked Studies. Then He Cherry-Picked His Own. appeared first on Reason Magazine.
