It's time to award the prize for best headline about the awarded of the prize for best physics by the Nobel committee. And by "best", I mean in the can't-look-away-from-the-train-wreck sense.
Background: as you may have heard, the Nobel Prize for Physics was announced, and surprising no one, it honoured the discovery of the Higgs boson (just the theorists though--no love for the experimentalists who, you know, actually found the thing).
Anyway, there were obviously lots of headlines on this issue, but I hearby award the following one from The Register, a British technology news website, for the honour of best headline, (by which I mean worst headline):
"Brit boson boffin Higgs bags Nobel with eponymous deiton"
I don't even know where to begin.
Scratch that, I'll begin with the words. "Boffin" seems to be some sort of British word for expert. That's fine, but the headline implies that Higgs is an expert on "bosons". Bosons, for those who don't know, are one class of particles; the other class is Fermions. Every single particle or collection of particles (which includes, well, everything) can be classed as either a boson or a fermion. Higgs isn't really a boson specialist, though--the category is so wide it's hard to know what that would even mean. Rather he used quantum field theory to predict a new particle, which happened to be a boson.
"Deiton" appears to be a new word coined by the good people at the Register. No explanation is given but it would seem to refer to the fact that the Higgs boson has sometimes been called the "God particle".
"Eponymous" seems like a strange choice for a context in which clarity is presumably important, especially when paired with the made-up "deiton".
Beyond the actual words, though, there's a bigger issue here, and it's conceptual. The issue is whether a headline exists to inform, to set context, and to draw in interested viewers; or to show off the cleverness of the headline writer. It seems like the Register opts for the latter.
So, an open letter to the Register (because they totally read this blog...):
Dear Register,
Please be aware that a truly clever headline is one that sets up the reader for what is to come in an accurate, concise, and clear way.
Also, as a rule of thumb, headlines should probably not contain words you just made up. Just saying.
Sincerely,
Everyone who cares about science communication
Showing posts with label sciheadlines. Show all posts
Showing posts with label sciheadlines. Show all posts
Wednesday, 9 October 2013
Tuesday, 13 August 2013
The "link" Between Autism and Induced Labour Could Be Just Noise (and in any case is being reported poorly)
Again with the autism links. This time, a study out of Duke University (which is unfortunately paywalled) showed a statistical link between autism and induced or augmented childbirth.
There are, of course, a few qualifying comments. The study authors note that their result doesn't imply that induced or augmented childbirth causes autism, even some of the time. There could be other factors at work. As a hypothetical example, if there was an unknown foetal developmental issue that both caused an increased risk of autism and interfered with the signalling that starts labour (which is still not well understood), such an issue would explain the study results--especially since the authors did not separately consider mothers who had labour induced because they went past term and mothers who had labour induced at term or before for other reasons. Most importantly, the authors stress, in such hypothetical case it would be useless from the point of view of autism prevention and downright harmful from the point of view of general maternal health for parents to refuse a medically recommended induction on the basis of this study.
There's another issue that the authors don't address, and it's worth mentioning: the study could just be picking up noise. The study reports a number of different results for various models, but they all are just barely significant. For example, the odds of an induced only (as opposed to induced or augmented) baby developing autism are 10% larger than that of an un-induced childbirth--with a confidence interval of 9%. 10+/-9 isn't very precise; the confidence interval is almost as large as the effect. Further, the confidence intervals reported are 95% confidence intervals, meaning there is a 1 in 20 chance of seeing an effect that large by chance.
A 1 in 20 chance is fine for most studies (though, if you think about the hundreds of studies published each year, it means an appalling number are wrong through sheer randomness). It's dangerous, though, in look-back studies of this type. I've written before about the problems here, but it's worth summarizing again. The key point is that if researchers are working on a problem with many possible connections, 1 in 20 suddenly seems quite poor. The question that needs to be asked is, how many other connections with autism were, at the start of the study, as likely as that between autism and induced labour?
Or to phrase it another way, was there some reason to think a priori that induced labour was more likely to be linked to autism than, say, breastfeeding vs formula, or the mother's food choice, or early life air quality, or lack of any number of vitamins, or abundance of any number of vitamins, or certain types of stimulation, etc? If there's not, than 1 in 20 isn't very good. This would be obvious if you actually tested all the possibilities: if you did a study that looked at 40 different possible causes for autism and came back with a positive result for one of them, with a 1 in 20 chance that the result was just due to random fluctuations, that obviously wouldn't mean anything. Not checking the other 39 possibilities doesn't make your statistics better.
Of course, we know that look-back studies like this are the most popular type reported in the media, and we've seen that even the study authors were either unaware or ignored the statistics of such studies. So it's not surprising, though it's still disappointing, to see all of the all of the nuance and caveats go out the window in the way this study has been reported.
There are, of course, a few qualifying comments. The study authors note that their result doesn't imply that induced or augmented childbirth causes autism, even some of the time. There could be other factors at work. As a hypothetical example, if there was an unknown foetal developmental issue that both caused an increased risk of autism and interfered with the signalling that starts labour (which is still not well understood), such an issue would explain the study results--especially since the authors did not separately consider mothers who had labour induced because they went past term and mothers who had labour induced at term or before for other reasons. Most importantly, the authors stress, in such hypothetical case it would be useless from the point of view of autism prevention and downright harmful from the point of view of general maternal health for parents to refuse a medically recommended induction on the basis of this study.
There's another issue that the authors don't address, and it's worth mentioning: the study could just be picking up noise. The study reports a number of different results for various models, but they all are just barely significant. For example, the odds of an induced only (as opposed to induced or augmented) baby developing autism are 10% larger than that of an un-induced childbirth--with a confidence interval of 9%. 10+/-9 isn't very precise; the confidence interval is almost as large as the effect. Further, the confidence intervals reported are 95% confidence intervals, meaning there is a 1 in 20 chance of seeing an effect that large by chance.
A 1 in 20 chance is fine for most studies (though, if you think about the hundreds of studies published each year, it means an appalling number are wrong through sheer randomness). It's dangerous, though, in look-back studies of this type. I've written before about the problems here, but it's worth summarizing again. The key point is that if researchers are working on a problem with many possible connections, 1 in 20 suddenly seems quite poor. The question that needs to be asked is, how many other connections with autism were, at the start of the study, as likely as that between autism and induced labour?
Or to phrase it another way, was there some reason to think a priori that induced labour was more likely to be linked to autism than, say, breastfeeding vs formula, or the mother's food choice, or early life air quality, or lack of any number of vitamins, or abundance of any number of vitamins, or certain types of stimulation, etc? If there's not, than 1 in 20 isn't very good. This would be obvious if you actually tested all the possibilities: if you did a study that looked at 40 different possible causes for autism and came back with a positive result for one of them, with a 1 in 20 chance that the result was just due to random fluctuations, that obviously wouldn't mean anything. Not checking the other 39 possibilities doesn't make your statistics better.
Of course, we know that look-back studies like this are the most popular type reported in the media, and we've seen that even the study authors were either unaware or ignored the statistics of such studies. So it's not surprising, though it's still disappointing, to see all of the all of the nuance and caveats go out the window in the way this study has been reported.
“Pregnant women who have procedures
to induce or encourage labor might have an increased risk of bearing
a child with autism, according to a new study,” reads the opening
line of the Wall Street Journal's article. Readers who finish the
article will learn that at best this a gross simplification, but
assuming that readers who are both busy and lacking a science
background will realize that the opening sentence here is at odds
with the quote is either a naive estimation of the audience or a deliberate confusion for the sake of an attention-grabbing hook.
The write-up in the NY Daily Mail has this helpful caption below their related picture: "Among autistic boys in a new study, one third of mothers had labor induced or hastened, compared to 29% of boys without autism." This is head-scratching on multiple levels: most obviously, why compare a percent with a fraction? People hate fractions. Second, they're reporting the numbers in the form of a probability the mother had induced labour given that child has autism. But this is entirely backwards, since what we actually want is the probability a child gets autism given that mother had induced labour, relative to the probability a child gets autism under any circumstance. The former is a needlessly confusing way of reporting the numbers, given that the latter form is available in the study, and is what's actually discussed in the study's conclusions.
(As an aside, the NY Daily Mail also wins the look-back study reporting award, with the following "Related" links scattered throughout their article: "STUDY LINKS IVF TO SMALL RISK OF MENTAL DISABILITY"; "ANOREXIC GIRLS ALSO HAVE AUTISTIC TRAITS: STUDY"; "HIGH LEVELS OF AIR POLLUTION LINKED TO AUTISM RISK".)
We also have the BBC News report. I should mention here that the BBC News story was the most responsible of the ones I looked at in stressing the limitations of the study and the importance of consulting with your doctor about delivery decisions. In it, though, we learn that "Children whose mothers needed drugs to start give birth are slightly more likely to have autism, US researchers say." Beyond the caveat noted above, that this actually could be just noise, there's an additional, subtle problem. Suppose the statement itself is true. Then is the statement, "If you have an induced childbirth, your child is slightly more likely to have autism," true? The answer is no.
Another example is helpful here. Suppose you and your spouse both have dark hair. Suppose also that your parents both had dark hair, and their parents, all dark hair stretching back to the dawn of recorded history. Suppose this is true of your spouse as well. Now let's imagine that the two of you move to Sweden together for work, and while you're there you have your first child. As you're being taken to the delivery room, the doctor says to you, "Did you know that 85% of children born in Sweden are blond? Since your child is a child born in Sweden, there's an 85% chance he or she will be blond."
Obviously this is ridiculous; moving to another country doesn't change your DNA, and if neither you nor your spouse is carrying blond genes, your baby isn't going to get them, no matter where you live. So even though it's true that 85% of children born in Sweden are blond1, and it's true that your child will be born in Sweden, it's not true that your child has an 85% chance of being born blond.
The exact same structure carries over: Even if it's true that children whose mothers had induced childbirth are 10% more likely to develop autism, and even if it's true that you had an induced childbirth, it is not necessarily true that your child is 10% more likely to develop autism.
This last point needs to be stressed, because it would be tragic if any mothers or infants were harmed out of a belief that inducing labour increased the chance of the child developing autism. I will most likely come back to it in another blog post, because the conflation of general odds with individual odds is at the heart of many of the most controversial misunderstandings we face.
In the mean time, while this blog doesn't exactly have the readership of the NY Daily Mail, I'll do what I can to spread what should be the take-home message here: at the moment there is no reason to believe that YOUR child is at a greater risk of autism if YOU have an induced labour. It's a message that would be clearer if news outlets and study authors were more focussed on getting the message right than on getting it out.
1Full disclosure: I have no idea what the actual rate of blond babies is in Sweden. It's just an example.
The write-up in the NY Daily Mail has this helpful caption below their related picture: "Among autistic boys in a new study, one third of mothers had labor induced or hastened, compared to 29% of boys without autism." This is head-scratching on multiple levels: most obviously, why compare a percent with a fraction? People hate fractions. Second, they're reporting the numbers in the form of a probability the mother had induced labour given that child has autism. But this is entirely backwards, since what we actually want is the probability a child gets autism given that mother had induced labour, relative to the probability a child gets autism under any circumstance. The former is a needlessly confusing way of reporting the numbers, given that the latter form is available in the study, and is what's actually discussed in the study's conclusions.
(As an aside, the NY Daily Mail also wins the look-back study reporting award, with the following "Related" links scattered throughout their article: "STUDY LINKS IVF TO SMALL RISK OF MENTAL DISABILITY"; "ANOREXIC GIRLS ALSO HAVE AUTISTIC TRAITS: STUDY"; "HIGH LEVELS OF AIR POLLUTION LINKED TO AUTISM RISK".)
We also have the BBC News report. I should mention here that the BBC News story was the most responsible of the ones I looked at in stressing the limitations of the study and the importance of consulting with your doctor about delivery decisions. In it, though, we learn that "Children whose mothers needed drugs to start give birth are slightly more likely to have autism, US researchers say." Beyond the caveat noted above, that this actually could be just noise, there's an additional, subtle problem. Suppose the statement itself is true. Then is the statement, "If you have an induced childbirth, your child is slightly more likely to have autism," true? The answer is no.
Another example is helpful here. Suppose you and your spouse both have dark hair. Suppose also that your parents both had dark hair, and their parents, all dark hair stretching back to the dawn of recorded history. Suppose this is true of your spouse as well. Now let's imagine that the two of you move to Sweden together for work, and while you're there you have your first child. As you're being taken to the delivery room, the doctor says to you, "Did you know that 85% of children born in Sweden are blond? Since your child is a child born in Sweden, there's an 85% chance he or she will be blond."
Obviously this is ridiculous; moving to another country doesn't change your DNA, and if neither you nor your spouse is carrying blond genes, your baby isn't going to get them, no matter where you live. So even though it's true that 85% of children born in Sweden are blond1, and it's true that your child will be born in Sweden, it's not true that your child has an 85% chance of being born blond.
The exact same structure carries over: Even if it's true that children whose mothers had induced childbirth are 10% more likely to develop autism, and even if it's true that you had an induced childbirth, it is not necessarily true that your child is 10% more likely to develop autism.
This last point needs to be stressed, because it would be tragic if any mothers or infants were harmed out of a belief that inducing labour increased the chance of the child developing autism. I will most likely come back to it in another blog post, because the conflation of general odds with individual odds is at the heart of many of the most controversial misunderstandings we face.
In the mean time, while this blog doesn't exactly have the readership of the NY Daily Mail, I'll do what I can to spread what should be the take-home message here: at the moment there is no reason to believe that YOUR child is at a greater risk of autism if YOU have an induced labour. It's a message that would be clearer if news outlets and study authors were more focussed on getting the message right than on getting it out.
1Full disclosure: I have no idea what the actual rate of blond babies is in Sweden. It's just an example.
Friday, 26 July 2013
A particularly bad headline
From Gizmodo: Scientists Just Discovered a New Force That's Stronger Than Gravity
Okay, so many things wrong with this headline. To start with, gravity is the weakest of the four fundamental forces, so "stronger than gravity" applies to just about everything. But that's not the biggest problem here. The headline, by pairing "new force" with "gravity", seems to suggest that someone has discovered a new fundamental force. This isn't what happened.
Quick overview: there are four fundamental forces. In order of strength they are: Gravity, Electromagnetism, Weak Nuclear Force, Strong Nuclear Force. Gravity is weak, but because it only ever adds (nothing has "negative" mass), it ends up being powerful on large scales, like planets and stars. The Weak Nuclear Force is a little obscure, but as the name suggests is involved in nuclear processes like atomic decay. The Strong Nuclear Force is what holds nuclei together, and it's what makes nuclear weapons so powerful.
Everything else is electromagnetism. Friction, contact forces, air pressure, water pressure, everything other than gravity that you experience can ultimately be traced back to electromagnetism in its various forms.
The "New Force" in this article is electromagnetism. The researchers looked at the effect of blackbody radiation (which is a type of electromagnetic radiation) on hydrogen atoms in an astrophysical context. (Here's the article; it's behind a paywall though.) They found that it could create an attractive force between the atoms which hadn't been appreciated before. But keep in mind that what they have found is a new way in which the electromagnetic force is expressed, not a "New Force" in the fundamental force sense.
Onto the "Stronger Than Gravity" part. The researchers noted that this blackbody force they discovered could be more important than gravity in the context of star formation. In a nebula, there's a lot of hydrogen, but it's really spread out--it's waaay less dense than air. (Star Trek has led many people astray by depicting nebulae as clouds that ships can hide in. Look at a picture of a nebula. In most of them you can see stars through them, even though they're light-years across. Anything with visibility measured in trillions of kilometres is a bad hiding place.) Somehow, over time the hydrogen in a nebula manages to coalesce into a star, with an enormous density. Exactly how this happens is a subject of ongoing research, and this new blackbody force could play an important role in that. Because at the densities the nebula starts at, the gravitational forces involved are super, super, super small. So even thought this blackbody force is super, super small, it could still be important.
It is not, though, "Stronger Than Gravity" in an every-day sense. You will not be levitated by blackbody forces. They will not explain dark matter or the expansion of the universe.
It's starting to sound like I might be down on this research, but I'm really not. It's a novel idea that is important to an area of ongoing research in astrophysics. That's basically the goal when you write a journal article. The headline writers at Gizmodo simply did a horrible job summarizing it. Good research deserves a better headline than this.
Okay, so many things wrong with this headline. To start with, gravity is the weakest of the four fundamental forces, so "stronger than gravity" applies to just about everything. But that's not the biggest problem here. The headline, by pairing "new force" with "gravity", seems to suggest that someone has discovered a new fundamental force. This isn't what happened.
Quick overview: there are four fundamental forces. In order of strength they are: Gravity, Electromagnetism, Weak Nuclear Force, Strong Nuclear Force. Gravity is weak, but because it only ever adds (nothing has "negative" mass), it ends up being powerful on large scales, like planets and stars. The Weak Nuclear Force is a little obscure, but as the name suggests is involved in nuclear processes like atomic decay. The Strong Nuclear Force is what holds nuclei together, and it's what makes nuclear weapons so powerful.
Everything else is electromagnetism. Friction, contact forces, air pressure, water pressure, everything other than gravity that you experience can ultimately be traced back to electromagnetism in its various forms.
The "New Force" in this article is electromagnetism. The researchers looked at the effect of blackbody radiation (which is a type of electromagnetic radiation) on hydrogen atoms in an astrophysical context. (Here's the article; it's behind a paywall though.) They found that it could create an attractive force between the atoms which hadn't been appreciated before. But keep in mind that what they have found is a new way in which the electromagnetic force is expressed, not a "New Force" in the fundamental force sense.
Onto the "Stronger Than Gravity" part. The researchers noted that this blackbody force they discovered could be more important than gravity in the context of star formation. In a nebula, there's a lot of hydrogen, but it's really spread out--it's waaay less dense than air. (Star Trek has led many people astray by depicting nebulae as clouds that ships can hide in. Look at a picture of a nebula. In most of them you can see stars through them, even though they're light-years across. Anything with visibility measured in trillions of kilometres is a bad hiding place.) Somehow, over time the hydrogen in a nebula manages to coalesce into a star, with an enormous density. Exactly how this happens is a subject of ongoing research, and this new blackbody force could play an important role in that. Because at the densities the nebula starts at, the gravitational forces involved are super, super, super small. So even thought this blackbody force is super, super small, it could still be important.
It is not, though, "Stronger Than Gravity" in an every-day sense. You will not be levitated by blackbody forces. They will not explain dark matter or the expansion of the universe.
It's starting to sound like I might be down on this research, but I'm really not. It's a novel idea that is important to an area of ongoing research in astrophysics. That's basically the goal when you write a journal article. The headline writers at Gizmodo simply did a horrible job summarizing it. Good research deserves a better headline than this.
Tuesday, 23 July 2013
Headlines in (Social) Science: Gender, Politics, and Unreviewed Findings
Often when I see a science story in the news the first thing I do is look up the related research article. That way I can see what was actually done, and evaluate, if not the detailed methods, at least the overall scientific logic of the article. If it's particularly new or controversial I sometimes bookmark the article so I can come back later and see what else has been published in response.
Of course, this requires that there be an article to look at. Since I'm at a university, paywalls aren't a problem, but even my university library subscription doesn't get me access to articles that haven't been published yet. Or even accepted. Which brings us to the story at the centre of today's post: a study funded by the Economic and Social Research Council, a British government funding agency, authored primarily by James Curran, director of the Goldsmiths Leverhulme Media Research Centre at the University of London.
According to various headlines, this study showed that:
Across the world, women know less about politics than men
Women know less about politics than men, study finds (that goes for Canada, too)
Women, especially in Canada, are more ignorant of politics and current affairs than men, says UK research
Study: Women Know Less About Politics Than Men
Did the study actually show this? To answer that we need to take a close look at the details of the research. As part of a CBC radio interview here, University of Calgary prof Melanee Thomas points out that there are a lot of ways these types of studies can be misleading. She brings up the important point that there can be biases in what is defined as political knowledge. Often "politics" is restricted in definition to so-called hard news: trade, military issues and conflicts, economic and budget issues, etc. Which these are certainly important, the list often leaves out other issues that are inarguably political: health, education, and all manner of issues of local governance. It's hard to argue that health policy isn't a "political" issue, but it is often labelled "soft" news and stuck with the other "human interest" stories. (As an aside, "human interest" is an interesting term: exactly why should I care about anything that's not of interest to humans?)
The point is that the study may be biasing its result by asking questions in the domains that men, on average, know more about, and ignoring domains that are, by any objective definition, equally political and which women know more about.
Does this study fall into those pitfalls? We don't know, and neither does Melanee Thomas, because the study hasn't been released yet. It's not listed on James Curran's research website, and the ESRC site lists the work as "submitted". As anyone in academia knows, a lot can happen between "submitted" and "published".
There's a couple of things to address here. First, we have the headline magnification I've talked about before. Curran holds a press conference in which he summarizes his work; a journalist takes that work and turns it into an article; an editor takes that article and turns it into a headline. Based on other cases of this headline magnification, these levels may have resulted in headlines that bear little resemblance to what was actually shown.
But, and this is the second point, we don't know what was actually shown, because the research is unpublished. And this is where James Curran has, in my opinion, acted reprehensibly. He has used his position as an expert to promote a conclusion without allowing the underlying work to be scrutinized. It will, obviously, be scrutinized eventually, but by the time that happens the original news stories will months in the past--an eternity in the online news world. No one is going to prominently display a story that adds context and corrections to a relatively minor headline from a few months ago. So prof Curran is putting out a conclusion that can't be verified, but that adds to a narrative that has women as intrinsically less able than men in certain key areas.
I don't think that research should be subordinated to social mores or political considerations. What I'm arguing is that researchers have a responsibility to ensure that, within the best of their ability, their research is reported correctly and in context. Especially when it has the potential to further harm already marginalized groups. We've seen the harm that can be done with headlines such as "Vaccines cause Autism." The harm from "Women know less than men about politics" may not be as obvious, but that doesn't mean it's not there. For that reason, Curran had a responsibility to give as nuanced and context-filled report as he could, and to allow other researchers the chance to dispute his findings and provide their own insights. He did none of that.
Of course, this requires that there be an article to look at. Since I'm at a university, paywalls aren't a problem, but even my university library subscription doesn't get me access to articles that haven't been published yet. Or even accepted. Which brings us to the story at the centre of today's post: a study funded by the Economic and Social Research Council, a British government funding agency, authored primarily by James Curran, director of the Goldsmiths Leverhulme Media Research Centre at the University of London.
According to various headlines, this study showed that:
Across the world, women know less about politics than men
Women know less about politics than men, study finds (that goes for Canada, too)
Women, especially in Canada, are more ignorant of politics and current affairs than men, says UK research
Study: Women Know Less About Politics Than Men
Did the study actually show this? To answer that we need to take a close look at the details of the research. As part of a CBC radio interview here, University of Calgary prof Melanee Thomas points out that there are a lot of ways these types of studies can be misleading. She brings up the important point that there can be biases in what is defined as political knowledge. Often "politics" is restricted in definition to so-called hard news: trade, military issues and conflicts, economic and budget issues, etc. Which these are certainly important, the list often leaves out other issues that are inarguably political: health, education, and all manner of issues of local governance. It's hard to argue that health policy isn't a "political" issue, but it is often labelled "soft" news and stuck with the other "human interest" stories. (As an aside, "human interest" is an interesting term: exactly why should I care about anything that's not of interest to humans?)
The point is that the study may be biasing its result by asking questions in the domains that men, on average, know more about, and ignoring domains that are, by any objective definition, equally political and which women know more about.
Does this study fall into those pitfalls? We don't know, and neither does Melanee Thomas, because the study hasn't been released yet. It's not listed on James Curran's research website, and the ESRC site lists the work as "submitted". As anyone in academia knows, a lot can happen between "submitted" and "published".
There's a couple of things to address here. First, we have the headline magnification I've talked about before. Curran holds a press conference in which he summarizes his work; a journalist takes that work and turns it into an article; an editor takes that article and turns it into a headline. Based on other cases of this headline magnification, these levels may have resulted in headlines that bear little resemblance to what was actually shown.
But, and this is the second point, we don't know what was actually shown, because the research is unpublished. And this is where James Curran has, in my opinion, acted reprehensibly. He has used his position as an expert to promote a conclusion without allowing the underlying work to be scrutinized. It will, obviously, be scrutinized eventually, but by the time that happens the original news stories will months in the past--an eternity in the online news world. No one is going to prominently display a story that adds context and corrections to a relatively minor headline from a few months ago. So prof Curran is putting out a conclusion that can't be verified, but that adds to a narrative that has women as intrinsically less able than men in certain key areas.
I don't think that research should be subordinated to social mores or political considerations. What I'm arguing is that researchers have a responsibility to ensure that, within the best of their ability, their research is reported correctly and in context. Especially when it has the potential to further harm already marginalized groups. We've seen the harm that can be done with headlines such as "Vaccines cause Autism." The harm from "Women know less than men about politics" may not be as obvious, but that doesn't mean it's not there. For that reason, Curran had a responsibility to give as nuanced and context-filled report as he could, and to allow other researchers the chance to dispute his findings and provide their own insights. He did none of that.
Tuesday, 16 July 2013
Headlines in Science: Scientists say...
The last post in this series looked at a specific example of a bad headline. This time, I want to zoom out a little and focus on a class of headlines. Specifically, all those that include "Scientists claim," or "Scientists say," or any equivalent to this, in the title.
This is a dangerous construct. The implication is that all scientists, or at least all scientists in a given field, or at least a majority of scientists in a given field, agree with the statement in the rest of the headline. It's understandable when you consider headlines as a way of generating interest in an article. After all, no one cares if "Joe from accounting totally thinks he gained weight when he stopped sleeping." But give it a title like "Lack of sleep can make you fat, scientists claim" and now we've got something.
However interest grabbing it may be, problems arise when the implied support of the scientific community collides with that effect we've discussed before, wherein an editor summarizes (and sensationalizes) an article whose author was summarizing (and sensationalizing) a research project. A potentially misleading or flat-out wrong statement has now been given the weight of expert consensus.
To look at the "lack of sleep can make you fat" example, looking at the article reveals that the scientists didn't actually measure weight, or BMI, or waist size. And they didn't run the experiment long enough to even see a noticeable weight gain. What they did was measure the levels of a particular chemical in the body that is linked to a desire to eat. Now there's nothing wrong with doing that, but as with all research it's important to be clear on what was and wasn't shown.
The next big problem with the "scientists claim" construct is that very often it's applied to the findings of one researcher, or at most a small collaboration. While technically it's true that a paper with three authors is "scientists" saying something, headlines using this construct give the impression of consensus, not just a small group.
The example we've already looked at falls squarely in this category; it's a single group reporting one study they performed. This problem, though, appears to be rife. Searching "scientists claim" and "scientists say" in Google news on 16 July 2013 brought up, in addition to the sleep-fat story:
"Singing And Yoga Might Have Same Health Benefits, Scientists Claim"
"Global warming 'can be reversed', scientists claim"
"Earth had two moons, scientists claim"
"'There is no scientific consensus' on sea-level rise, say scientists"
In every one of these stories, upon actually reading the story you realize that each is based on one paper published by one research group. The research might be born out, and the headline may actually reflect scientific consensus in a few years (well, except the last one, which is a flatly disingenuous article from the climate-change deniers camp). But at the moment they're jumping the gun.
There's one last point I want to make here. Often "scientists claim" headlines do include a qualifier: might, may, can, etc. This in and of itself isn't a bad thing. But I don't think it lets editors off the hook for making the statements they then qualify. Now, I don't actually have any research backing up what I'm about to say (if anyone else knows of any, I'd love to hear about it!). But from personal experience and talking to other people, it seems like qualifiers are the first things forgotten when recalling a headline. I don't generally remember the exact wording, I remember the idea and I paraphrase it, which comes out something like "this study showed that if you get less sleep you gain weight." Qualifier gone.
For this particular problem, there's a pretty easy solution: stop using "scientists claim"! Or any other equivalent construct. At the very least, reserve it for statements that come out of large conferences designed to forge a consensus. But on the whole, editors, please just stop.
Maybe then I can stop losing sleep over terrible headlines. Which I heard was making me fat. It's true; scientists say so.
This is a dangerous construct. The implication is that all scientists, or at least all scientists in a given field, or at least a majority of scientists in a given field, agree with the statement in the rest of the headline. It's understandable when you consider headlines as a way of generating interest in an article. After all, no one cares if "Joe from accounting totally thinks he gained weight when he stopped sleeping." But give it a title like "Lack of sleep can make you fat, scientists claim" and now we've got something.
However interest grabbing it may be, problems arise when the implied support of the scientific community collides with that effect we've discussed before, wherein an editor summarizes (and sensationalizes) an article whose author was summarizing (and sensationalizing) a research project. A potentially misleading or flat-out wrong statement has now been given the weight of expert consensus.
To look at the "lack of sleep can make you fat" example, looking at the article reveals that the scientists didn't actually measure weight, or BMI, or waist size. And they didn't run the experiment long enough to even see a noticeable weight gain. What they did was measure the levels of a particular chemical in the body that is linked to a desire to eat. Now there's nothing wrong with doing that, but as with all research it's important to be clear on what was and wasn't shown.
The next big problem with the "scientists claim" construct is that very often it's applied to the findings of one researcher, or at most a small collaboration. While technically it's true that a paper with three authors is "scientists" saying something, headlines using this construct give the impression of consensus, not just a small group.
The example we've already looked at falls squarely in this category; it's a single group reporting one study they performed. This problem, though, appears to be rife. Searching "scientists claim" and "scientists say" in Google news on 16 July 2013 brought up, in addition to the sleep-fat story:
"Singing And Yoga Might Have Same Health Benefits, Scientists Claim"
"Global warming 'can be reversed', scientists claim"
"Earth had two moons, scientists claim"
"'There is no scientific consensus' on sea-level rise, say scientists"
In every one of these stories, upon actually reading the story you realize that each is based on one paper published by one research group. The research might be born out, and the headline may actually reflect scientific consensus in a few years (well, except the last one, which is a flatly disingenuous article from the climate-change deniers camp). But at the moment they're jumping the gun.
There's one last point I want to make here. Often "scientists claim" headlines do include a qualifier: might, may, can, etc. This in and of itself isn't a bad thing. But I don't think it lets editors off the hook for making the statements they then qualify. Now, I don't actually have any research backing up what I'm about to say (if anyone else knows of any, I'd love to hear about it!). But from personal experience and talking to other people, it seems like qualifiers are the first things forgotten when recalling a headline. I don't generally remember the exact wording, I remember the idea and I paraphrase it, which comes out something like "this study showed that if you get less sleep you gain weight." Qualifier gone.
For this particular problem, there's a pretty easy solution: stop using "scientists claim"! Or any other equivalent construct. At the very least, reserve it for statements that come out of large conferences designed to forge a consensus. But on the whole, editors, please just stop.
Maybe then I can stop losing sleep over terrible headlines. Which I heard was making me fat. It's true; scientists say so.
Saturday, 13 July 2013
Headlines in Science: io9 and Objectifying Women
This is the first post in a series on headlines in science communication. This one is from last year, but I'm going to talk about it because it's a particularly egregious example of a terrible headline.
One of the blogs I read is io9, a blog about science, technology, and nerdy things in general. Normally I enjoy frequenting this part of the internet. But in this case they participated in the all-too-common trend of headline inflation - that is, taking some research, adding an interpretation to it that wasn't in the original but that makes it more attention-grabbing, and then making that interpretation the headline. In this case, the research was this, an article (unfortunately behind a paywall) that sought to quantify the mental processes that drive the objectification of women's bodies.
I won't go into all the details, but basically the study looked at how likely people were to notice changes in specific body parts in pictures of men and women, and how easily they identified pictures they saw earlier from a picture of just one part of the person. It was a study looking at the way the minds of the participants worked, while assuming throughout that a) objectification is a bad thing, and b) studying the way the mind works could help us combat it.
Move on over to the Scientific American blog post covering the story. Here the bare facts are reported, and the post stresses that objectification is harmful, and points to an experiment that could lead to a better understanding of how to put your brain into a non-objectifying mode. But the author also wants to get at more than just what is in the study, so we get this quote:
There could be evolutionary reasons that men and women process female bodies differently, Gervais said, but because both genders do it, "the media is probably a prime suspect."
Note that the quote they got from the study author (Gervais) says "the media is probably a prime suspect," but the post author chose to place it in a context that implies that evolutionary reasons are also a possibility, something not mentioned at all in the research article. Obviously, I have no idea what their conversation was, but it seems like the journalist asked whether evolution could have hardwired us for this, and the study author gave a researcher's version of "no." Bear in mind that scientists get good reputations by not being wrong, so they tend to hedge any statements they make, especially ones on which they don't have conclusive data from multiple sources. The journalist here took that "no" and shaped it into a sentence that comes across as a "maybe..." It's a little sad, but not entirely unexpected.
Now we get to the io9 article. The headline is "Both men and women may be hardwired to objectify women's bodies". At least, it was (I'll get to that in a bit.) At this point, my question for io9 is, WHAT THE HELL?!?! With one headline they took an article whose goal was to help reduce objectification, and turned it into something that could be used as an excuse to keep doing it. The comments indicated that a number of people took it exactly as that. (One comment: "I'm staring at your titties because of science baby. Pure science." Presumably not the reaction the study author was hoping for.) I appreciate the fact that io9 wants to increase their page views, since that is how they generate revenue, but completely inverting the subject of research in a way that harms the people the research was trying to help so that more people are directed to their blog is despicable. There's no way to soften that or justify what they've done here.
Just in case that wasn't bad enough, the article title was changed, after a number of the comments pointed out that it was completely at odds with the research it was reporting. Now it reads, "Why both men and women's eyes are drawn to women's bodies." Perhaps no more accurate, but at least less offensive. (The original title is still in the web address). But, there's no mention in the article that this has been edited. So now, the numerous people who made comments suggesting that this was a sexist attempt to increase page views are left sounding like they over-reacted to a straightforward article. If you're going to correct a mistake on a thing like this, the least you could do is fess up to it. Not that would have undone the damage. Since io9 is a high volume blog, even someone who checks in once a day won't ever see the article again, unless they scroll down the sidebar looking for it. All in all, this amazing double-whammy might be the biggest science journalism fail I've seen in a long time.
This whole episode is a sad illustration of the process that creates terrible headlines, and the damage they can do. To start with, the science being reported on was an exploratory work; an initial study that will presumably be followed by others to given more nuance and context. Exploratory work often turns out to be incorrect, and when it is correct, the best interpretation is bounced around the research community, often for years, until the work is settled in context with other work and theories, and a story emerges. Any take-home messages that are suggested before then are, at best, the opinion of one researcher regarding the significance of their own work, or at worst, the opinion of one editor regarding work in a field they have only a passing knowledge of. This type of science needs to be considered especially carefully as it is especially prone to bad headlines.
The next step in this sad process is the multiple layers of sensationalizing. Here there were three levels: the original blog at Scientific American, the article at io9, then finally the headline for that article. Each level pushed the conclusions from the original research into more sensational, and less accurate, territory, until the point was completely lost.
Finally, the damage. Though the body of the article never makes the statement the headline does, it's pretty clear from the comments that a number of people assumed the headline was the conclusion of the research being reported on. And why not? Isn't that the point of a headline? It's difficult to quantify this, but it's safe to say that, for each person that took the time to comment on the article, many more simply saw the headline and added it to their internal "facts I know" database.
The process by which this headline went wrong suggests some things that could be done. First, read the original research. I know that paywalls are a problem for many science enthusiasts, but there's really no excuse for a professional journalist to not get a copy of the original article. Reporting on a report on some research is bound to introduce distortions. Second (and this will be a theme), the headline should be tossed back as far as possible. What I mean by this is that in the worst case scenario, the headline is okayed by the journalist writing the article, and in the best case, by the scientist actually doing the research. This won't always be realistic given news timelines, but throwing the headline back as far as possible to double-check it can only help.
So, thank you, io9, for that wonderful illustration of how to be completely terrible at making headlines. I sincerely hope the rest of the articles in this little series have far less to work with.
One of the blogs I read is io9, a blog about science, technology, and nerdy things in general. Normally I enjoy frequenting this part of the internet. But in this case they participated in the all-too-common trend of headline inflation - that is, taking some research, adding an interpretation to it that wasn't in the original but that makes it more attention-grabbing, and then making that interpretation the headline. In this case, the research was this, an article (unfortunately behind a paywall) that sought to quantify the mental processes that drive the objectification of women's bodies.
I won't go into all the details, but basically the study looked at how likely people were to notice changes in specific body parts in pictures of men and women, and how easily they identified pictures they saw earlier from a picture of just one part of the person. It was a study looking at the way the minds of the participants worked, while assuming throughout that a) objectification is a bad thing, and b) studying the way the mind works could help us combat it.
Move on over to the Scientific American blog post covering the story. Here the bare facts are reported, and the post stresses that objectification is harmful, and points to an experiment that could lead to a better understanding of how to put your brain into a non-objectifying mode. But the author also wants to get at more than just what is in the study, so we get this quote:
There could be evolutionary reasons that men and women process female bodies differently, Gervais said, but because both genders do it, "the media is probably a prime suspect."
Note that the quote they got from the study author (Gervais) says "the media is probably a prime suspect," but the post author chose to place it in a context that implies that evolutionary reasons are also a possibility, something not mentioned at all in the research article. Obviously, I have no idea what their conversation was, but it seems like the journalist asked whether evolution could have hardwired us for this, and the study author gave a researcher's version of "no." Bear in mind that scientists get good reputations by not being wrong, so they tend to hedge any statements they make, especially ones on which they don't have conclusive data from multiple sources. The journalist here took that "no" and shaped it into a sentence that comes across as a "maybe..." It's a little sad, but not entirely unexpected.
Now we get to the io9 article. The headline is "Both men and women may be hardwired to objectify women's bodies". At least, it was (I'll get to that in a bit.) At this point, my question for io9 is, WHAT THE HELL?!?! With one headline they took an article whose goal was to help reduce objectification, and turned it into something that could be used as an excuse to keep doing it. The comments indicated that a number of people took it exactly as that. (One comment: "I'm staring at your titties because of science baby. Pure science." Presumably not the reaction the study author was hoping for.) I appreciate the fact that io9 wants to increase their page views, since that is how they generate revenue, but completely inverting the subject of research in a way that harms the people the research was trying to help so that more people are directed to their blog is despicable. There's no way to soften that or justify what they've done here.
Just in case that wasn't bad enough, the article title was changed, after a number of the comments pointed out that it was completely at odds with the research it was reporting. Now it reads, "Why both men and women's eyes are drawn to women's bodies." Perhaps no more accurate, but at least less offensive. (The original title is still in the web address). But, there's no mention in the article that this has been edited. So now, the numerous people who made comments suggesting that this was a sexist attempt to increase page views are left sounding like they over-reacted to a straightforward article. If you're going to correct a mistake on a thing like this, the least you could do is fess up to it. Not that would have undone the damage. Since io9 is a high volume blog, even someone who checks in once a day won't ever see the article again, unless they scroll down the sidebar looking for it. All in all, this amazing double-whammy might be the biggest science journalism fail I've seen in a long time.
This whole episode is a sad illustration of the process that creates terrible headlines, and the damage they can do. To start with, the science being reported on was an exploratory work; an initial study that will presumably be followed by others to given more nuance and context. Exploratory work often turns out to be incorrect, and when it is correct, the best interpretation is bounced around the research community, often for years, until the work is settled in context with other work and theories, and a story emerges. Any take-home messages that are suggested before then are, at best, the opinion of one researcher regarding the significance of their own work, or at worst, the opinion of one editor regarding work in a field they have only a passing knowledge of. This type of science needs to be considered especially carefully as it is especially prone to bad headlines.
The next step in this sad process is the multiple layers of sensationalizing. Here there were three levels: the original blog at Scientific American, the article at io9, then finally the headline for that article. Each level pushed the conclusions from the original research into more sensational, and less accurate, territory, until the point was completely lost.
Finally, the damage. Though the body of the article never makes the statement the headline does, it's pretty clear from the comments that a number of people assumed the headline was the conclusion of the research being reported on. And why not? Isn't that the point of a headline? It's difficult to quantify this, but it's safe to say that, for each person that took the time to comment on the article, many more simply saw the headline and added it to their internal "facts I know" database.
The process by which this headline went wrong suggests some things that could be done. First, read the original research. I know that paywalls are a problem for many science enthusiasts, but there's really no excuse for a professional journalist to not get a copy of the original article. Reporting on a report on some research is bound to introduce distortions. Second (and this will be a theme), the headline should be tossed back as far as possible. What I mean by this is that in the worst case scenario, the headline is okayed by the journalist writing the article, and in the best case, by the scientist actually doing the research. This won't always be realistic given news timelines, but throwing the headline back as far as possible to double-check it can only help.
So, thank you, io9, for that wonderful illustration of how to be completely terrible at making headlines. I sincerely hope the rest of the articles in this little series have far less to work with.
Scientists say Headlines Generally Terrible
A headline serves as the title for a news story, but obviously it's more than that. It serves as a guide, letting the reader know what to expect; it forms a context for what they are about to read; and it may be the only part of the article many people look at. Which is to say it's important that a headline be as accurate as you can be in one line.
Unfortunately, headlines often aren't written by the author of the attached article--they're added by an editor or someone else involved with the publishing process. So there's a double layer of understanding to surmount: the science journalist understanding the material and communicating it clearly in an article, then an editor understanding the article and making a good headline. Sadly, the point of a piece of research or a discovery is often lost in these two translations.
It gets even worse when the subject matter is controversial. Now, instead of just two layers of understanding going into the headline, there are also two layers of sensationalism. I'm not trying to say here that science journalists are irresponsible tabloid writers, just that by nature they look at a story and ask, "What's the excitement here? How can we spice up this story?" Then the editor looks over the story and asks, "What's the excitement here? How can we spice up this headline?" Two layers of this, and it's no wonder that the headline often ends up an extreme exaggeration of the science it purports to describe.
It's helpful to look at some examples to see where headlines can go wrong, and what the damage can be. With that in mind, this post is merely the introduction to what I hope will be an ongoing series looking at headlines in science articles. For the most part it will be focussed on where they go wrong, although if I come across anything that strikes me as a particularly good headline for a subtle or difficult topic, I'll post about that too.
Often I see articles pointing out problems with a format, or institution, or society, without any suggestions as to what can be done. This can be useful, but obviously has its limits. So I'm going to try to think about how headlines can be better generated as I post each article about them. Hopefully I'll leave you with not just "This is a problem," but "This is a problem, and here's how it could have been done better."
So, now for some terrible headlines!
Unfortunately, headlines often aren't written by the author of the attached article--they're added by an editor or someone else involved with the publishing process. So there's a double layer of understanding to surmount: the science journalist understanding the material and communicating it clearly in an article, then an editor understanding the article and making a good headline. Sadly, the point of a piece of research or a discovery is often lost in these two translations.
It gets even worse when the subject matter is controversial. Now, instead of just two layers of understanding going into the headline, there are also two layers of sensationalism. I'm not trying to say here that science journalists are irresponsible tabloid writers, just that by nature they look at a story and ask, "What's the excitement here? How can we spice up this story?" Then the editor looks over the story and asks, "What's the excitement here? How can we spice up this headline?" Two layers of this, and it's no wonder that the headline often ends up an extreme exaggeration of the science it purports to describe.
It's helpful to look at some examples to see where headlines can go wrong, and what the damage can be. With that in mind, this post is merely the introduction to what I hope will be an ongoing series looking at headlines in science articles. For the most part it will be focussed on where they go wrong, although if I come across anything that strikes me as a particularly good headline for a subtle or difficult topic, I'll post about that too.
Often I see articles pointing out problems with a format, or institution, or society, without any suggestions as to what can be done. This can be useful, but obviously has its limits. So I'm going to try to think about how headlines can be better generated as I post each article about them. Hopefully I'll leave you with not just "This is a problem," but "This is a problem, and here's how it could have been done better."
So, now for some terrible headlines!
Subscribe to:
Posts (Atom)