It’s PEM Academic Meeting Season and John Oliver Rails on Bad Science

Whether you’re asking the questions, teaching trainees, or incorporating evidence into your daily practice, many of us in PEM rely on sound research. With many recovering from the nerdy hangover of the recent Pediatric Academic Societies meeting and this week’s Society for Academic Emergency Medicine meeting activities, I have been thinking about how to digest the all of the posters, platform presentations, and calls for further studies on every topic.

John Oliver—Daily Show correspondent turned incredulous HBO news anchor—dedicated a 20-minute segment on how science is interpreted and reported today. And spoiler alert: He was not pleased. He pointed out the varying outlandish scientific report such as how much your dog hates hugs and the importance of flatulence. It’s a humorous yet sobering reminder of the common pitfalls of interpreting science and how the incentives in academic publishing can be distorting.

Watching the segment is entertaining. But if you’ve become too busy for humor and joy, this is what I took away from this.

  • Studies—especially small ones—need validation before they are genaralizeable yet the incentives for performing these studies are small. Researchers get kudos for new discoveries yet are ignored or even mocked (behind closed doors, of course) for validating existing results and “not adding anything new.“
  • People may cherry pick findings they want to hear. I am guilty of this, as I am a big proponent of exercising very little based on just the headline of a New York Times article.
  • If you read the headline, you might do the same.

  • Statistics may be misleading. Practice change should not be based on glancing at the p-values without looking at the methods and actual results. While sometimes painful, journal clubs and critical reading may be a remedy for this when done well.
  • The media may mean well but can simplify and distort conclusions. Some journalists are as guilty as some academics of reading abstracts and press releases before taking a few shortcuts in an effort to get in front of the news cycle.

Many of these flaws in science have always existed although the growth of digital media also amplifies many of them. How will PEM—which many of us consider a young and innovative specialty—handle this delicate academic balance?

Sonny Tat

Sonny Tat

Assistant Clinical Professor at University of California at San Francisco
Sonny's academic interests are in team performance in resuscitation, developing learning platforms for medical education, and transitioning academics to the digital age. Sonny created PEMAcademy, a collaborative regional blog for PEM education and is the Editor-In-Chief of the PEMNetwork Blog. His hobbies include photography and cooking.
Sonny Tat
Sonny Tat

Sonny Tat

Sonny's academic interests are in team performance in resuscitation, developing learning platforms for medical education, and transitioning academics to the digital age. Sonny created PEMAcademy, a collaborative regional blog for PEM education and is the Editor-In-Chief of the PEMNetwork Blog. His hobbies include photography and cooking.