News articles on biomedical studies have since the year 2000 used more hyperbolic headlines and more frequently omitted replication statements, a team of University of Bordeaux researchers found. Estelle Dumas-Mallet, Andy Smith, Thomas Boraud and François Gonon analysed over 400 news stories on biomedical research, published globally between 1988 and 2009.

First the authors selected a sample of over 5 000 biomedical studies, and then narrowed it down to initial studies that were covered by at least two English language newspaper articles. This left the authors with 40 studies and 426 newspaper articles written about them. Finally, the team analysed the news articles’ contents: whether the findings were honestly described as uncertain, whether the headlines tried to “hype” the findings, and whether the study’s author had been interviewed.

Initial biomedical findings are always tentative, and they must be validated by subsequent replication, the authors remind. In a previous research they had found that most (51.3 per cent) biomedical findings reported by the media were later invalidated by replication studies. Thus the authors were interested to see whether the need for replication was mentioned in newspaper articles about new studies.

Less than a quarter of the studied articles (21.4 per cent) included a “replication statement”, while more articles (24.6 per cent) claimed the results were “robust”. As expected, replication statements were thrice as common when the study’s author had been interviewed.

Little over third of all headlines (35.2 per cent) were evaluated as “hyped” by the authors. Further, hyped headlines correlate negatively with the presence of replication statements: when the tentative nature of the results is mentioned in the news text, the headline is also less likely to exaggerate the findings.

The authors also compared the results from before and after the year 2000. They found that replication statements were more common, and hyperbolic headlines more rare, on the last millennium. Since most of the sample’s articles were published in just four countries, the authors compared them to each other. According to this analysis, British science news were the most misleading (most hyped headlines and least replication statements) and American news the best, with Australia and Canada falling between the two.

“Sadly, our study confirms that most newspaper articles do not inform the public about the uncertainty inherent in initial studies”, the authors conclude. While hyping up studies may provide short-term benefits in newspaper readership and publicity to research, the long-term effects may be negative, Dumas-Mallet and others warn.

The article “Scientific Uncertainty in the Press” was published by the journal Science Communication. It is available online on the publisher’s website (abstract free).

Picture: Untitled by Adrian Malec, licence CC0 1.0.