Tuesday, 16 July 2013

Headlines in Science: Scientists say...

The last post in this series looked at a specific example of a bad headline. This time, I want to zoom out a little and focus on a class of headlines. Specifically, all those that include "Scientists claim," or "Scientists say," or any equivalent to this, in the title.

This is a dangerous construct. The implication is that all scientists, or at least all scientists in a given field, or at least a majority of scientists in a given field, agree with the statement in the rest of the headline. It's understandable when you consider headlines as a way of generating interest in an article. After all, no one cares if "Joe from accounting totally thinks he gained weight when he stopped sleeping." But give it a title like "Lack of sleep can make you fat, scientists claim" and now we've got something.

However interest grabbing it may be, problems arise when the implied support of the scientific community collides with that effect we've discussed before, wherein an editor summarizes (and sensationalizes) an article whose author was summarizing (and sensationalizing) a research project. A potentially misleading or flat-out wrong statement has now been given the weight of expert consensus.

To look at the "lack of sleep can make you fat" example, looking at the article reveals that the scientists didn't actually measure weight, or BMI, or waist size. And they didn't run the experiment long enough to even see a noticeable weight gain. What they did was measure the levels of a particular chemical in the body that is linked to a desire to eat. Now there's nothing wrong with doing that, but as with all research it's important to be clear on what was and wasn't shown.

The next big problem with the "scientists claim" construct is that very often it's applied to the findings of one researcher, or at most a small collaboration. While technically it's true that a paper with three authors is "scientists" saying something, headlines using this construct give the impression of consensus, not just a small group.

The example we've already looked at falls squarely in this category; it's a single group reporting one study they performed. This problem, though, appears to be rife. Searching "scientists claim" and "scientists say" in Google news on 16 July 2013 brought up, in addition to the sleep-fat story:

"Singing And Yoga Might Have Same Health Benefits, Scientists Claim"

"Global warming 'can be reversed', scientists claim"

"Earth had two moons, scientists claim"

"'There is no scientific consensus' on sea-level rise, say scientists"

In every one of these stories, upon actually reading the story you realize that each is based on one paper published by one research group. The research might be born out, and the headline may actually reflect scientific consensus in a few years (well, except the last one, which is a flatly disingenuous article from the climate-change deniers camp). But at the moment they're jumping the gun.

There's one last point I want to make here. Often "scientists claim" headlines do include a qualifier: might, may, can, etc. This in and of itself isn't a bad thing. But I don't think it lets editors off the hook for making the statements they then qualify. Now, I don't actually have any research backing up what I'm about to say (if anyone else knows of any, I'd love to hear about it!). But from personal experience and talking to other people, it seems like qualifiers are the first things forgotten when recalling a headline. I don't generally remember the exact wording, I remember the idea and I paraphrase it, which comes out something like "this study showed that if you get less sleep you gain weight." Qualifier gone.

For this particular problem, there's a pretty easy solution: stop using "scientists claim"! Or any other equivalent construct. At the very least, reserve it for statements that come out of large conferences designed to forge a consensus. But on the whole, editors, please just stop.

Maybe then I can stop losing sleep over terrible headlines. Which I heard was making me fat. It's true; scientists say so.

No comments:

Post a Comment