Image by Carla Pedret©

Statistics are everywhere, and journalists – especially data journalists – should know how to interpret them. Numbers are like any other source, and journalists should have the knowledge to find out when they are misleading.

How can journalists improve their use of statistics? What are the most common mistakes? As statisticians are the best ones to answer these questions, I started a discussion in StatsUserNet, an online community hosted by the Royal Statistical Society. With their permission, I publish in this post a digest of tips, tricks and claims shared in the conversation. Since some of the participants don’t want to be identified, I anonymised their comments.

Statistics as an absolute truth

One of the most repeated comments in the discussion is that journalists lack numeracy skills and maths know-how.

According to David Sibert, many journalists think that statistics represent the truth and can be reported without further investigation.

Political parties and corporations use statistics to defend or justify particular interests. Hence, it is indispensable that journalists analyse numbers as deeply as they analyse words.

David Sibert adds that it is socially unacceptable to admit that one is poor at reading or spelling, but people will proudly boast: “I’m no good at maths”. Journalists could actually use their position to help stop numerical illiteracy.

Magnifying risks

Statisticians are also concerned about how journalists convert low-risk threats into headlines.

For Maria Ottati, the main problem is media talk about risks without taking into account the amount of people affected. For instance, the headline “Eating food X doubles the risk of developing cancer type Y” could be correct, but the article must explain that “cancer type Y is extremely rare and the risk of developing it is 1 in 100 million, so that a 50% increase is pretty meaningless”.

Iain Brennan agrees. As an example, he mentions a headline a police colleague showed him: “2,000 calls to police unanswered”. It is an eye-catching headline, but the number mentioned represents only about 0.5% of all non-emergency calls. This point was explained in the penultimate paragraph of the story.

The highest, the lowest and the outliers: journalists’ favourites

Paul Kingswell notes how journalists analyse a trend through the highest/lowest rates nationally, but without contextualising the data.  

That happened when media published Infant mortality rates. North Warwickshire Borough was shown to have the joint highest rate in England. However, that was based on just 5 deaths and hence, as noted by the ONS, the rate was of “low reliability”.

The statisticians that joined the discussion encourage journalists to analyse datasets as a whole and to be careful when choosing the top or the bottom value of a spreadsheet.

Percentage change and percentage points

Another repeated mistake is the confusion between percentage change and a change in percentage points.

Related to percentages, some comments highlight the overuse of percentages as opposed to fractions, with often misleading results. The suggestion is to use both of them carefully to avoid mistakes.

When working with percentages, it is always better to round a number: instead of writing 99.999%, better say 100%.

Conceptual problems

Ian Heath points out how some concepts that actually don’t exist are commonly used in the media.

In sports interviews, for example, some journalists refer to the “law of averages”, which does not exist.

Growth is usually called “exponential growth” to signify large growth. However, all growth is usually exponential, even, say, a constant 10% change.

Ian Heath notes that there is a tendency to show absurd formulae on a board to show a “boffin” at work, whereas it would be regarded as ignorant and illiterate to show someone writing bad grammar, spelling and punctuation.

This picture, published at The Guardian, is a good example:

Books and free resources to learn statistics

If you want to know more about statistics, there is a wide range of books, free courses and resources online that can help you. Some examples:

• Karen Hurrell recommends the Royal Statistical Society’s free courses for journalists.
• David Sibert suggests as a mandatory reading for journalists “Risk: The Science and Politics of Fear” by Dan Gardner
• Iain Brennan mentions the book “The tiger that isn’t” by Michael Blastland and Andrew Dilnot with lots of examples with common mistakes.
• “How to lie with statistics” by Darrell Huff is also a classic book about the topic.
• “A mathematician reads the newspaper” by John Allen Paulos is another book worth reading.

Bonus tip

Statistics can be tricky to understand and to explain. When dealing with them, don’t forget the famous quote by Aaron Levenstein:

 “Statistics are like a bikini. What they reveal is suggestive, but what they conceal is vital”.