2008 - Bad Science Read online

  Ben Goldacre

  Bad Science

  2008

  Guardian columnist Dr Ben Goldacre takes us on a hilarious, invigorating and informative journey through the bad science we’re fed by the worst of the hacks and the quacks! When Dr Ben Goldacre saw someone on daytime TV dipping her feet in an ‘Aqua Detox’ footbath, releasing her toxins into the water and turning it brown, he thought he’d try the same at home. ‘Like some kind of Johnny Ball cum Witchfinder General’, using his girlfriend’s Barbie doll, he gently passed an electrical current through the warm salt water. It turned brown. In his words: ‘before my very eyes, the world’s first Detox Barbie was sat, with her feet in a pool of brown sludge, purged of a weekend’s immorality.’ Dr Ben Goldacre is the author of the ‘Bad Science’ column in the Guardian and his book is about all the ‘bad science’ we are constantly bombarded with in the media and in advertising. At a time when science is used to prove everything and nothing, everyone has their own ‘bad science’ moments—from the useless pie-chart on the back of cereal packets to the use of the word ‘visibly’ in cosmetics ads.This book will help people to quantify their instincts—that a lot of the so-called ‘science’ which appears in the media and in advertising is just wrong or misleading. Satirical and amusing—and unafraid to expose the ridiculous—it provides the reader with the facts they need to differentiate the good from the bad. Full of spleen, this is a hilarious, invigorating and informative journey through the world of ‘bad science’.

  INTRODUCTION

  Let me tell you how bad things have become. Children are routinely being taught—by their own teachers, in thousands of British state schools—that if they wiggle their head up and down it will increase blood flow to the frontal lobes, thus improving concentration; that rubbing their fingers together in a special sciencey way will improve ‘energy flow’ through the body; that there is no water in processed food; and that holding water on their tongue will hydrate the brain directly through the roof of the mouth, all as part of a special exercise programme called ‘Brain Gym’. We will devote some time to these beliefs and, more importantly, the buffoons in our education system who endorse them.

  But this book is not a collection of trivial absurdities. It follows a natural crescendo, from the foolishness of quacks, via the credence they are given in the mainstream media, through the tricks of the £30 billion food supplements industry, the evils of the £300 billion pharmaceuticals industry, the tragedy of science reporting, and on to cases where people have wound up in prison, derided, or dead, simply through the poor understanding of statistics and evidence that pervades our society.

  At the time of C.P. Snow’s famous lecture on the ‘Two Cultures’ of science and the humanities half a century ago, arts graduates simply ignored us. Today, scientists and doctors find themselves outnumbered and outgunned by vast armies of individuals who feel entitled to pass judgement on matters of evidence—an admirable aspiration—without troubling themselves to obtain a basic understanding of the issues.

  At school you were taught about chemicals in test tubes, equations to describe motion, and maybe something on photosynthesis—about which more later—but in all likelihood you were taught nothing about death, risk, statistics, and the science of what will kill or cure you. The hole in our culture is gaping: evidence-based medicine, the ultimate applied science, contains some of the cleverest ideas from die past two centuries, it has saved millions of lives, but there has never once been a single exhibit on the subject in London’s Science Museum.

  This is not for a lack of interest. We are obsessed with health—half of all science stories in the media are medical—and are repeatedly bombarded with sciencey-sounding claims and stories. But as you will see, we get our information from the very people who have repeatedly demonstrated themselves to be incapable of reading, interpreting and bearing reliable witness to the scientific evidence.

  Before we get started, let me map out the territory.

  Firsdy, we will look at what it means to do an experiment, to see the results with your own eyes, and judge whether they fit with a given theory, or whether an alternative is more compelling. You may find these early steps childish and patronising—the examples are certainly refreshingly absurd—but they have all been promoted credulously and with great authority in the mainstream media. We will look at the attraction of sciencey-sounding stories about our bodies, and the confusion they can cause.

  Then we will move on to homeopathy, not because it’s important or dangerous—it’s not—but because it is the perfect model for teaching evidence-based medicine: homeopathy pills are, after all, empty little sugar pills which seem to work, and so they embody everything you need to know about ‘fair tests’ of a treatment, and how we can be misled into thinking that any intervention is more effective than it really is. You will learn all there is to know about how to do a trial properly, and how to spot a bad one. Hiding in the background is the placebo effect, probably the most fascinating and misunderstood aspect of human healing, which goes far beyond a mere sugar pill: it is counterintuitive, it is strange, it is the true story of mind-body healing, and it is far more interesting than any made-up nonsense about therapeutic quantum energy patterns. We will review the evidence on its power, and you will draw your own conclusions.

  Then we move on to the bigger fish. Nutritionists are alternative therapists, but have somehow managed to brand themselves as men and women of science. Their errors are much more interesting than those of the homeopaths, because they have a grain of real science to them, and that makes them not only more interesting, but also more dangerous, because the real threat from cranks is not that their customers might die—there is the odd case, although it seems crass to harp on about them—but that they systematically undermine the public’s understanding of the very nature of evidence.

  We will see the rhetorical sleights of hand and amateurish errors that have led to you being repeatedly misled about food and nutrition, and how this new industry acts as a distraction from the genuine lifestyle risk factors for ill health, as well as its more subtle but equally alarming impact on the way we see ourselves and our bodies, specifically in the widespread move to medicalise social and political problems, to conceive of them in a reductionist, biomedical framework, and peddle commodifiable solutions, particularly in the form of pills and faddish diets. I will show you evidence that a vanguard of startling wrongness is entering British universities, alongside genuine academic research into nutrition. This is also the section where you will find the nation’s favourite doctor, Gillian McKeith, PhD. Then we apply these same tools to proper medicine, and see the tricks used by the pharmaceutical industry to pull the wool over the eyes of doctors and patients.

  Next we will examine how the media promote the public misunderstanding of science, their single-minded passion for pointless non-stories, and their basic misunderstandings of statistics and evidence, which illustrate the very core of why we do science: to prevent ourselves from being misled by our own atomised experiences and prejudices. Finally, in the part of the book I find most worrying, we will see how people in positions of great power, who should know better, still commit basic errors, with grave consequences; and we will see how the media’s cynical distortion of evidence in two specific health scares reached dangerous and frankly grotesque extremes. It’s your job to notice, as we go, how incredibly prevalent this stuff is, but also to think what you might do about it.

  You cannot reason people out of positions they didn’t reason themselves into. But by the end of this book you’ll have the tools to win—or at least understand—any argument you choose to initiate, whether it’s on miracle cures, MMR, the evils of big pharma, the likelihood of a given vegetable preventing cancer, the dumbing
down of science reporting, dubious health scares, the merits of anecdotal evidence, the relationship between body and mind, the science of irrationality, the medicalisation of everyday life, and more. You’ll have seen the evidence behind some very popular deceptions, but along the way you’ll also have picked up everything useful there is to know about research, levels of evidence, bias, statistics (relax), the history of science, anti-science movements and quackery, as well as falling over just some of the amazing stories that the natural sciences can tell us about the world along the way.

  It won’t be even slightly difficult, because this is the only science lesson where I can guarantee that the people making the stupid mistakes won’t be you. And if, by the end, you reckon you might still disagree with me, then I offer you this: you’ll still be wrong, but you’ll be wrong with a lot more panache and flair than you could possibly manage right now.

  Ben Goldacre

  July 2008

  1 Matter

  I spend a lot of time talking to people who disagree with me—I would go so far as to say that it’s my favourite leisure activity—and repeatedly I meet individuals who are eager to share their views on science despite the fact that they have never done an experiment. They have never tested an idea for themselves, using their own hands; or seen the results of that test, using their own eyes; and they have never thought carefully about what those results mean for the idea they are testing, using their own brain. To these people ‘science’ is a monolith, a mystery, and an authority, rather than a method.

  Dismantling our early, more outrageous pseudoscientific claims is an excellent way to learn the basics of science, partly because science is largely about disproving theories, but also because the lack of scientific knowledge among mirade-cure artistes, marketers and journalists gives us some very simple ideas to test Their knowledge of sdence is rudimentary, so as well as making basic errors of reasoning, they also rely on notions like magnetism, oxygen, water, ‘energy’ and toxins: ideas from GCSE-level science, and all very much within the realm of kitchen chemistry.

  Detox and the theatre of goo

  Since you’ll want your first experiment to be authentically messy, we’ll start with detox. Aqua Detox is a detox footbath, one of many similar products. It has been promoted uncritically in some very embarrassing articles in the Telegraph, the Mirror, the Sunday Times, GQ magazine and various TV shows. Here is a taster from the Mirror.

  We sent Alex for a new treatment called Aqua Detox which releases toxins before your eyes. Alex says: ‘I place my feet in a bowl of water, while therapist Mirka pours salt drops in an ionising unit, which will adjust the bio-energetic field of the water and encourage my body to discharge toxins. The water changes colour as the toxins are released. After half an hour, the water’s turned red…she gets our photographer Karen to give it a go. She gets a bowl of brown bubbles. Mirka diagnoses an overloaded liver and lymph—Karen needs to drink less alcohol and more water. Wow, I feel virtuous!’

  The hypothesis from these companies is very clear: your body is full of ‘toxins’, whatever those may be; your feet are filled with special ‘pores’ (discovered by ancient Chinese scientists, no less); you put your feet in the bath, the toxins are extracted, and the water goes brown. Is the brown in the water because of the toxins? Or is that merely theatre?

  One way to test this is to go along and have an Aqua Detox treatment yourself at a health spa, beauty salon, or any of me thousands of places they are available online, and take your feet out of the bath when the therapist leaves the room. If the water goes brown without your feet in it, then it wasn’t your feet or your toxins that did it. That is a controlled experiment: everything is the same in both conditions, except for the presence or absence of your feet.

  There are disadvantages with this experimental method (and there is an important lesson here, that we must often weigh up the benefits and practicalities of different forms of research, which will become important in later chapters). From a practical perspective, the ‘feet out’ experiment involves subterfuge, which may make you uncomfortable. But it is also expensive: one session of Aqua Detox will cost more man the components to build your own detox device, a perfect model of the real one.

  You will need:

  One car battery charger

  Two large nails

  Kitchen salt

  Warm water

  One Barbie doll

  A full analytic laboratory (optional)

  This experiment involves electricity and water. In a world of hurricane hunters and volcanologists, we must accept that everyone sets their own level of risk tolerance. You might well give yourself a nasty electric shock if you perform this experiment at home, and it could easily blow the wiring in your house. It is not safe, but it is in some sense relevant to your understanding of MMR, homeopathy, post-modernist critiques of science and the evils of big pharma. Do not build it.

  When you switch your Barbie Detox machine on, you will see that the water goes brown, due to a very simple process called electrolysis: the iron electrodes rust, essentially, and the brown rust goes into the water. But there is something more happening in there, something you might half-remember from chemistry at school. There is salt in the water. The proper scientific term for household salt is ‘sodium chloride’: in solution, this means that there are chloride ions floating around, which have a negative charge (and sodium ions, which have a positive charge). The red connector on your car battery charger is a ‘positive electrode’, and here, negatively charged electrons are stolen away from the negatively charged chloride ions, resulting in the production of free chlorine gas.

  So chlorine gas is given off by the Barbie Detox bath, and indeed by the Aqua Detox footbath; and the people who use this product have elegantly woven that distinctive chlorine aroma into their story: it’s the chemicals, they explain; it’s the chlorine coming out of your body, from all the plastic packaging on your food, and all those years bathing in chemical swimming pools. ‘It has been interesting to see the colour of the water change and smell the chlorine leaving my body,’ says one testimonial for the similar product Emerald Detox. At another sales site: ‘The first time she tried the Q2 [Energy Spa], her business partner said his eyes were burning from all the chlorine, that was coming out of her, leftover from her childhood and early adulthood.’ All that chemically chlorine gas that has accumulated in your body over the years. It’s a frightening thought.

  But there is something else we need to check. Are there toxins in the water? Here we encounter a new problem: what do they mean by toxin? I’ve asked the manufacturers of many detox products this question time and again, but they demur. They wave their hands, they talk about stressful modern lifestyles, they talk about pollution, they talk about junk food, but they will not tell me the name of a single chemical which I can measure. ‘What toxins are being extracted from the body with your treatment?’ I ask. ‘Tell me what is in the water, and I will look for it in a laboratory.’ I have never been given an answer.

  After much of their hedging and fudging, I chose two chemicals pretty much at random: creatinine and urea. These are common breakdown products from your body’s metabolism, and your kidneys get rid of them in urine. Through a friend, I went for a genuine Aqua Detox treatment, took a sample of brown water, and used the disproportionately state-of-the-art analytic facilities of St Mary’s Hospital in London to hunt for these two chemical ‘toxins’. There were no toxins in the water. Just lots of brown, rusty iron.

  Now, with findings like these, scientists might take a step back, and revise their ideas about what is going on with the footbaths. We don’t really expect the manufacturers to do that, but what they say in response to these findings is very interesting, at least to me, because it sets up a pattern that we will see repeated throughout the world of pseudoscience: instead of addressing the criticisms, or embracing the new findings in a new model, they seem to shift the goalposts and retreat, crucially, into untestable positions.

  Some o
f them now deny that toxins come out in the footbath (which would stop me measuring them): your body is somehow informed that it is time to release toxins in the normal way—whatever that is, and whatever the toxins are—only more so. Some of them now admit that the water goes a bit brown without your feet in it, but ‘not as much’. Many of them tell lengthy stories about the ‘bioenergetic field’, which they say cannot be measured, except by how well you are feeling. All of them talk about how stressful modern life is.

  That may well be true. But it has nothing to do with their foot bath, which is all about theatre: and theatre is the common theme for all detox products, as we will see. On with the brown goo.

  Ear candles

  You might think that Hopi Ear Candles are an easy target. But their efficacy has still been cheerfully promoted by the Independent, the Observer and the BBC, to name just a few respected news outlets. Since these people are the authoritative purveyors of scientific information, I’ll let the BBC explain how these hollow wax tubes will detox your body:

  The candles work by vaporising their ingredients once lit, causing convectional air flow towards the first chamber of the ear. The candle creates a mild suction which lets the vapours gently massage the eardrum and auditory canal. Once the candle is placed in the ear it forms a seal which enables wax and other impurities to be drawn out of the ear.

  The proof comes when you open a candle up, and discover that it is filled with a familiar waxy orange substance, which must surely be earwax. If you’d like to test this yourself, you will need: an ear, a clothes peg, some Blu Tack, a dusty floor, some scissors, and two ear candles. I recommend OTOSAN because of their strapline (‘The ear is the gateway to the soul’).

  If you light one ear candle, and hold it over some dust, you will find little evidence of any suction. Before you rush to publish your finding in a peer-reviewed academic journal, someone has beaten you to it: a paper published in the medical journal Laryngoscope used expensive tympanometry equipment and found—as you have—that ear candles exert no suction. There is no truth to the claim that doctors dismiss alternative therapies out of hand.