How numbers can compel, charm and mislead us

An Erasmus MC press release headline on Tuesday, 25 September regarding a study about smokers and ex-smokers blares: ‘Screening can save thousands of lives’. With much fanfare, the academic medical centre announced that conducting a CT scan every few years among smokers and ex-smokers leads to earlier detection of lung cancer. This reduces the chance of dying from this disease.

But following a critical article in de Volkskrant the next day, the new, slightly less bombastic headline now reads: ‘Lung cancer screening with CT prevents deaths’. Because it turns out that while there were significantly fewer deaths from lung cancer in the screened group, the total number of deaths – from 934 to 904 – is not a significant difference.

But you could even cast doubt on that, says Sanne Blauw, about a week later in a cafe at Amsterdam Central Station. The reason for the interview is her book ‘The best-selling book of all time (with this title)’ slated to hit the shelves later this month. Her engaging writing style reveals how numbers can compel, charm and mislead us. In short, she’s trying to point out to us mortals that the barrage of numbers thrown about by manufacturers, scientists, politicians and media should not be thoughtlessly accepted as the truth. Or put more accurately, they shouldn’t be considered true or untrue at face value. Instead, they should be met with the appropriate amount of scepticism.

A good example is the Erasmus MC press release about screening smokers and ex-smokers. “The errors in this press release tick all the boxes in terms of what I write about in my book,” says Blauw. “First of all, the party issuing the press release has a huge vested interest regarding the result, even if it’s only the large amount of money involved in the study. Second, the information was publicised at a convention, while no justification of the research was given and the research hadn’t even been published yet. Believe me, this is completely unacceptable. And the way the numbers were collected was also not representative, and there was an unsupported extrapolation that thousands of deaths could be avoided. There are numerous errors to be found in their claim.”

However, Blauw is not surprised by such cases. Since swapping academia for a career as a journalist four years ago, she now works as a ‘numbers sleuth’ for de Correspondent. Since then she has seen her share of questionable research figures and dubious polls cross her path. Sometimes it’s a brewery-sponsored study into the effects of alcohol on health. Or intelligence tests with biased questions and experiments using atypical random sampling. All these topics are discussed in her book.

In itself, there’s nothing novel about the subject matter. Whenever numbers are used to explain something, there is always scope to manipulate the findings, and someone will write to expose this manipulation. A good example is Darrel Huff’s brilliant ‘How to lie with statistics’ published in 1954. Blauw agrees that this is indeed the case, but she feels it can’t hurt to refresh people’s memory with a slightly more user-friendly work on the topic. In fact, since Huff’s book was published, essential psychological insights have been unearthed regarding the relationship between humans and numbers. These insights have been incorporated in ‘The best-selling book of all time (with this title)’.

“These misunderstandings persist, so I felt it was time for a new book,” explains Blauw. She can also get quite worked up when she sees what can go wrong, and in this sense, she’s a woman on a mission. “There’s often a lack of knowledge among those on the receiving end of numbers. A lot of people have math anxiety and find numbers intimidating. At the same time they are also guilty of misinterpreting numbers because they are guided by their gut feelings. The book illustrates this perfectly in a case revealing how political affiliation actually influences how numbers are interpreted. In the US, Republicans frequently take the position that stricter gun control measures don’t work. This was also the case among a group of Republicans who were presented with numbers that could be interpreted to support the opposite of their position. The same could be observed among Democrats: they were more likely to see a reduction in crime even when this wasn’t supported by the numbers.

Blauw is now aware of why we let ourselves be misled by numbers: “We assume that numbers are objective evidence, but that’s not the case. We think that numbers are very precise and leave no room for doubt or grey areas. But of course that’s not true.” That’s why Blauw’s book opens with a scene where she recounts an interview with the Bolivian cleaner Juanita. The interview is for her Economics doctoral research into happiness and income disparity.

“It turns out the survey I concocted on the eleventh floor of the Woudestein campus wasn’t really adequate. And that’s when I first got the creeping feeling that life is more complex than numbers. Here I was, sitting across from a flesh-and-blood person who was saying things I couldn’t express with numbers.”

Blauw slowly came to realise she was more interested in the stories behind the numbers rather than the numbers themselves. Remarkable, in light of her fascination with numbers, even as a child. She initially wanted to study medicine, but upon seeing a classmate’s copy of the econometrists’ bible ‘Calculus’, she opted to study Econometrics. Perhaps the seed that led to her now deciphering rather than manipulating numbers was sown in her interview with Juanita all those years ago.

In her book she uses William Bruce Cameron’s adage ‘Not everything that counts, can be counted. Not everything that can be counted, counts’ to show that what you measure is contrived, based on bias, and often ultimately expressed in just a single number. This is the case whether we’re talking about someone’s IQ, global warming, or the probability percentage of dying from a particular disease.

This opens the floodgates for a series of ‘mistakes’. The research conditions or the questions are unsound, the composition of the research population for the study is not representative or large enough, margins of uncertainty are overlooked (deliberately or unintentionally), and last but not least, the party producing or disseminating the numbers has a vested interest in the findings.

All things considered, by her own account and in keeping with her personality, Blauw doesn’t fall into the trap of rejecting numbers altogether, in spite of the fallacies, gut feelings and vested interest that can almost always be found lurking in the background. “I’m not anti-numbers or anything like that,” she says. A sentence in her book says numbers are no more than a window to reality. “But even so, the window pane isn’t always fully transparent.”