On Tara Westover’s “Educated.”

On Tara Westover’s “Educated.”

A man in my poetry class recently told me, “Ugh, cocaine is awful.  You use some, you’ll want some more, but I hate it.  It makes me such a jerk.  I mean, I’m not like this, I’m never like this, but if I’m on coke, I’m like, bitch, you best make yourself useful around here.”

Cocaine has a reputation as a fun party drug, but nobody in jail has anything nice to say about it.  And it’s not that they’re down on drugs in general – that same man told me:

“Meth?  Meth is great – you should never try it.”

And then he explained the social niceties of trying to shoot up in the home of a friend who was recently diagnosed with HIV.  This friend was apparently cavalierly sloppy with needles:

“Like, blood was spurting, and I was scooting back thinking, like, god, I wish I was anywhere but here … “

#

Glasseelskils_0European eels are endangered.  They swim huge distances to complete their life cycles – hatching at sea, undertaking a voracious (oft cannibalistic) quest up rivers, then returning to their birthplaces to spawn – and have been thwarted by hydroelectric dams blocking their migration, and the tendency of an insatiable terrestrial ape to catch and consume huge numbers of their kind.

Now these eels face another obstacle: they must complete their voyages while blitzed on cocaine.  European governments dump drugs into the sea to “destroy” them, but that’s not how water works.  The drugs are still there.  The eels get high.

#

576px-Robert_Louis_Stevenson_by_Henry_Walter_Barnett_bwAccording to popular legend, Robert Louis Stevenson was very sick before he wrote his Strange Case of Dr. Jekyll and Mr. Hyde.

Stevenson had tried many cures; all had failed.  Then his spouse bought cocaine.  This worked.  Suddenly Stevenson could write again.  In three days, he composed his novel.

When he read the first draft to his spouse, she said it didn’t seem sufficiently allegorical.  So Stevenson flung the pages into the fire and began again.  In three more days, he’d composed the version of the story that we know today.

#

Dr. Jekyll was a fine man.  On drugs, he became a monster.

#

IMG_5233When our first child was just shy of two years old, she liked to wear a green long-sleeve with a picture of The Incredible Hulk rampaging.  She’d pull it from her drawer; I’d say, “Oh, you want to wear your Hulk Smash shirt today?”

One day, I asked her, “N., why does Hulk want to smash?”

She looked down at the picture, then back up to me.  First she signed the word hungry.

“Oh, Hulk wants something to eat?”

She shook her head.  No, that didn’t sound quite right.  She looked down again, then made another sign, banging her hands together for the word shoes.

“Hulk is upset because he has no shoes?”

She bobbed her head yes.  No shoes.  That would make her rage, too.

#

Once, some runners on the local high school cross country team asked me who would win in a fight, Superman or the Hulk.  They’d reached a detente after one claimed that the Hulk was unstoppable when enraged (“… and nothing calms him down except his girlfriend.”), and the other argued that Superman could turn back time until the moment before Hulk had gotten angry, then smash him.

4600608792_bc43de9c17_z.jpg
Superman vs. Hulk by JD Hancock on Flickr.

I demurred.  I don’t think Superman is a very interesting hero, and the Hulk is interesting only in campaigns, not battles.  I like the idea of a hero who might go berserk and accidentally thwart his own plans, but a single bout of wrestling isn’t like that.  I think it’s more compelling to consider his constant risk of hurting the people that he loves.

#

fullsizeoutput_12In Educated, Tara Westover writes beautifully about the horrors of living with the Hulk.  Her early years were controlled by a father in the throes of extreme paranoia and delusions of grandeur:

Fourteen years after the incident with the Weavers, I would sit in a university classroom and listen to a professor of psychology describe something called bipolar disorder.  Until that moment I had never heard of mental illness.  I knew people could go crazy – they’d wear dead cats on their heads or fall in love with a turnip – but the notion that a person could be functional, lucid, persuasive, and something could still be wrong, had never occurred to me.

The professor recited facts in a dull, earthy voice: the average age of onset is twenty-five; there may be no symptoms before then. 

The irony was that if Dad was bipolar – or had any of a dozen disorders that might explain his behavior – the same paranoia that was a symptom of the illness would prevent its ever being diagnosed and treated.  No one would ever know. 

Because her father was at war with the federal government, Westover never went to school.  Her birth went undocumented – she didn’t have a certificate that would’ve allowed her to enroll until years later, and even then wound up with a hodgepodge of documents that listed slightly differing names and birthdates. 

And her father needed money, because he was frantically stockpiling food and ammunition.  He needed solar panels (back when they were much more expensive than today) because the power grid was going to cut out after Y2K.

As one of God’s soldiers, he needed to build an ark.  Or tank.  Arsenal.  Whatever.

This constant hustle for money led Westover’s father to subject his children to incredible dangers.  There might be a safe way to do a job, but if the risky way could save two minutes, the man put his kids’ lives on the line.  Westover was forced to ride up to a trailer inside a bin filled with two thousand pounds of scrap iron.  When her leg got caught and she couldn’t jump out, her father still dumped the bucket.  Westover tumbled nearly twenty feet to the ground.  And this was lucky.  If she’d fallen a few inches to the other side, she would’ve been crushed by all that iron. 

Her brothers were injured even more grievously at her father’s hands.

For instance, a brother’s clothes caught fire while he was working with his father.  In Westover’s recollection, the father then lifted his burnt son into the cab of a truck and made him drive home alone.  Only the ten-year-old Westover was there to help him, so she put her brother’s burnt leg inside a garbage can full of ice water.

She thought:

If Dad had been with Luke on the mountain, he would have brought him to the house, would have treated the burn.  Dad was away on a job somewhere, that’s why Luke had had to get himself down the mountain.  Why his leg had been treated by a ten-year-old.  Why it had ended up in a garbage can.

Except that she then realizes that her father must have been with her brother.  He must have been there in order to put out the fire; otherwise the whole mountainside would’ve burned. 

In a footnote, Westover adds: 

Since the writing of this story, I have spoken to Luke about the incident.  His account differs from both mine and Richard’s.  In Luke’s memory, Dad took Luke to the house, administered a homeopathic for shock, then put him in a tub of cold water, where he left him to go fight the fire.  This goes against my memory, and against Richard’s.  Still, perhaps our memories are in error.  Perhaps I found Luke in a tub, alone, rather than on the grass.  What everyone agrees upon, strangely, is that somehow Luke ended up on the front lawn, his leg in a garbage can.

Westover’s father was abusive, and he routinely convinced his children that their memories were in error, instead substituting his own (oft-illogical) versions of events, but he isn’t the Hulk in this story.  Yup, things get worse.  One of Westover’s brothers might suddenly snap and become Hyde.

Westover loved her older brother Shawn, but during an over-hasty job with their father, Shawn fell twelve feet, striking a concrete wall headfirst, and sustained a severe brain injury.  Instead of taking the kid to the hospital, their father propped him against a pickup truck and left him to sit in the hot sun.

His pupils were unevenly dilated.  His brain was bleeding.

Fifteen minutes later, Shawn wandered back to the worksite and started acting wild.  He screamed, flung his father, ran around leaping and howling.  The others tackled him – at which point his head again struck the concrete, hard – and called 911 for a helicopter to airlift him to the hospital.

It took some time for me to realize that although Shawn looked the same, he wasn’t.  He seemed lucid, but if you listened carefully his stories didn’t make sense.  They weren’t really stories at all, just one tangent after another. 

Worse, he was violent.  But unpredictably so.  At one moment, he and Westover might be laughing together.  At another moment, he’d twist her arm behind her back so brutally that she worried her wrist would break, call her a slut, and cram her face into a toilet bowl.  He hacked at the throat of his son’s pet dog with a five-inch knife blade while the animal howled, dying.  He called his sister and placidly explained his plans to visit her university and murder her.

In a lucid moment, he helped Westover install a massive deadbolt in her bedroom door, despite knowing that he was the only person she needed protection from.

# 

Tara_Westover_1+2-smallerAnd yet, Westover escaped.  Although she’d never set foot inside a classroom, she was accepted to Brigham Young University, where the consensus view of reality was much closer to her own.

Of course, she made a few stumbles.  Because she didn’t understand what course numbers signified, she enrolled as a freshman in an upper-level art history class.  Worse, she raised her hand to ask after the meaning of a word she didn’t recognize: Holocaust.

During one of my own classes, we were discussing poems from Charles Reznikoff’s Testimony when I mentioned that Reznikoff had also written about the Holocaust.

“Holocaust, what’s that?” a man asked. 

Unlike Westover, this man had grown up in an urban area.  But he’d stopped attending school when he was pretty young, and there’s a lot of cultural knowledge that we tend to take for granted.

I’d like to think that I handled the situation better than Westover’s professor.  Westover was shamed.  In our poetry class, we instead talked about how the word “holocaust” could be seen as offensive when used to describe the years during which members of the Nazi party murdered at least 6 million people, typically because their victims believed in Judaism.  The word “holocaust” originally meant a burnt offering for God, so Jewish leaders instead referred to this period of history with the Hebrew word for “catastrophe.”  Although even that phrasing seems off, because “catastrophe” generally evokes natural disaster, whereas the Holocaust was mass murder and torture on a scale comparable only to American slavery.  A purely human evil.

#

Westover became a historian.  After experiencing firsthand the nightmare of having her own account of reality constantly replaced by someone else’s version, she understood how powerful storytelling can be.  Educated is a beautiful book.  And, to my mind, a much more sensible depiction of unequal opportunity in the United States than J.D. Vance’s Hillbilly Elegy.

Westover recognizes how lucky she was to escape, and how narrowly she avoided the fate of her sister-in-law.  And Westover gives a powerful endorsement of government aid:

I began to experience the most powerful advantage of money: the ability to think of things besides money.  My professors came into focus, suddenly and sharply; it was as if before the grant I’d been looking at them through a blurred lens.  My textbooks began to make sense, and I found myself doing more than the required reading.

On Sci-Hub, the Napster of science.

On Sci-Hub, the Napster of science.

Here’s a story you’ve probably heard: the music industry was great until Napster came along and complete strangers could “share” their collections online and profits tanked.  Metallica went berserk suing their fans.  It was too late.  The industry has never been the same.napster

Sci-Hub has been called a Napster equivalent for scientific research papers, and the major publishing companies are suing to shut it down.  The neuroscience grad student who created it faces financial ruin.  The original website was quickly shuttered by a legal injunction, but the internet is a slippery place.  Now the same service is hosted outside U.S. jurisdiction.

[Note: between writing and posting this essay, Sci-Hub has lost another lawsuit requesting all such sites to be blocked by internet service providers.]

The outcomes of these lawsuits are a big deal.  Not just for the idealistic Kazakhstani grad student charged with millions in damages.  Academic publishers will do all they can to accentuate the parallels between Sci-Hub and Napster – and, look, nearly a quarter of my living relatives are professional musicians, so I realize how much damage was wrought by Napster’s culture of theft – but comparing research papers to pop songs is a rotten analogy.  Even if you’ve never wanted to read original research yet … even if you think – reasonably – that content producers should be paid, you should care about the open access movement.  Of which Sci-Hub is the most dramatic foray.

My own perspective changed after I did some ghostwriting for a pop medicine book.  Maybe you know the type: “Do you have SCARY DISEASE X?  It’ll get better if you take these nutritional supplements and do this type of yoga and buy these experimental home-use medical devices!”  Total hokum.  And yet, people buy these books.  So there I was, unhelpfully – quite possibly unethically – collaborating with a friend who’d been hired to ghostwrite a new one.

Central_core_disease_NADH_stainI read huge numbers of research papers and wrote chapters about treating this particular SCARY DISEASE with different foods, nutritional supplements, and off-label pharmaceuticals.  My sentences were riddled with un-truths.  The foods and drugs I described are exceedingly unlikely to benefit patients in any way.

Still, I found research papers purporting to have found benefits.  I dutifully described the results.  I focused on the sort of semi-farcical study that concludes, for instance, that cancer patients who drink sufficient quantities of green tea have reduced tumor growth, at which point newspapers announce that green tea is a “superfood” that cures cancer, at which point spurious claims get slathered all over the packaging.

Maybe nobody has written a paper (yet!) claiming that green tea ameliorates your particular SCARY DISEASE.  But there’s also turmeric, kale, fish oil, bittermelon, cranberries… I’m not sure any ingredient is so mundane that it won’t eventually be declared a superfood.  Toxoplasma gondii has been linked to schizophrenia, but low-level schizophrenia has been linked to creativity: will it be long before cat excrement is marketed as a superfood for budding artists?

cat-shit-2-flat-1.jpgAs it happens, enough people suffer from our book’s SCARY DISEASE that many low-quality studies exist.  I was able to write those chapters.  And then felt grim.  The things I’d written about food weren’t so bad, because although turmeric, coconut oil, and carpaccio won’t cure anybody, they won’t cause much harm either.  But the drugs?  They won’t help, and most have nasty side effects.

My words might mislead people into wasting money on unnecessary dietary supplements or, worse, causing serious damage with self-prescribed pharmaceuticals.  Patients might follow the book’s rotten advice instead of consulting with a trained medical professional.  I’d like to think that nobody would be foolish enough to trust that book – the ostensible author is probably even less qualified to have written that book than I am, because at least I have a Ph.D. in biochemistry from Stanford – but, based on the money being thrown around, somebody thinks it’ll sell.

And I helped.

Whoops.  Mea culpa, and all of that.

But I didn’t perpetrate my sins alone.  And I’m not just blaming the book’s publishers here.  After all, the spurious results I described came from real research papers, often written by professors at major universities, often published in legitimate scientific journals.

It’s crummy to concentrate all that slop in a slim pop medicine book, I agree, but isn’t it also crummy for all those spurious research papers to exist at all?

Maybe you’ve heard that various scientific fields suffer from a “replication crisis.”  There’s been coverage on John Oliver’s Last Week Tonight and in the New York Times about major failures in psychology and medicine.  Scientists write a paper claiming something happens, but that thing doesn’t happen in anyone else’s hands.  That’s if anyone even bothers to check.  Most of the time, nobody does.  Verifying someone else’s results won’t help researchers win grants, so it’s generally seen as a waste of time and money.

Still, the news coverage I’ve seen hasn’t stated the problem sufficiently bluntly.  Modern academic science is designed to be false.

This is tragic.  It’s part of why I chose not to stop working in the field.  I became a writer.  Of course, this led to my stint of ghostwriting, which… well, whoops.

Here’s how modern science works: most research is publishable only if it is “statistically significant.”  This means comparing any result to a “null hypothesis” – if you’re investigating the effect of green tea on cancer, the null hypothesis is simply “green tea does nothing” – then throwing out your results if you had more than a one in twenty chance to see what you did if the null hypothesis were true.

If you have a hundred patients, some of their tumors will shrink no matter what you do.  If you give everybody buckets of green tea and see the usual number of people improve, you shouldn’t claim that green tea saved them.

Here’s a graphic from Wikipedia to help:

pvalue1pvalue2

Logical enough.  But bad.  Why?  Because cancer is a SCARY DISEASE.  Far more than twenty people are studying it.  If twenty scientists each decide to test whether green tea reduces tumors, the “one in twenty” statistical test means that somebody from that set of scientists will probably see an above-average number of patients improve.  When you’re dealing with random chance, there are always flukes.  If twenty researchers all decided to flip four coins in a row, somebody would probably see all four come up heads – doesn’t mean that researcher did anything special.

Or, did you hear the news that high folate might be correlated with autism?  This study probably sounds legitimate – the lead scientist is a professor at Johns Hopkins, after all – but the result is quite unlikely to be real.  That scientist hasn’t written about folate previously, so my best guess (this new study is currently unpublished) is that pregnant women were tested for many different biomarkers, things like folate, iron, testosterone, and more, and then tracked to see whose children would develop autism.  If the researchers tested the concentrations of twenty different nutrients and hormones, of course they’d see one that appeared to correlate with autism.

[Edit: these findings were recently published.  Indeed, the data appear rather unconvincing, and the measurements for folate were made after the fact, using blood samples – it’s quite possible that other data was gathered but excluded from the published version of the study.]

This is not science.  But if you neglect to mention how many biomarkers you studied, and you retroactively concoct a conspiracy theory-esque narrative explaining why you were concerned about folate, it can do a fine job of masquerading as science.  At least long enough to win the next grant.

Which means that, even though the results of many of these studies are false, they get published.  When somebody checks twenty nutrients, one might appear to cause autism.  When twenty scientists study green tea and cancer, somebody might get results suggesting green tea does work.  Even if it doesn’t do a thing.

In our current system, though, only the mistaken researcher’s results get published.  Nobody knows that there were twenty tests.  The nineteen other biomarkers that were measured get left out of the final paper.  The nineteen researchers who found that green tea does nothing don’t publish anything.  Showing that a food doesn’t cure cancer?  How mundane.  Nobody wants to read that; publishers don’t want it in their journals.  But the single spurious result showing that green tea is a tumor-busting superfood?  That is exciting.  That study lands in a fancy journal and gets described in even briefer, more flattering language in the popular press.  Soon big-name computer CEOs are guzzling green tea instead of risking surgery or chemo.

I generally assume that the conclusions of research studies using this type of statistical testing are false.  And there’s more.  Data are often presented misleadingly.  Plenty of scientists are willing to test a pet theory many ways and report only the approach that “works,” not necessarily because they want to lie to people, but because it’s so easy to rationalize why the test you tried first (and second, and third…) was not quite right.  I worked in many laboratories over a decade and there were often results that everybody in the lab knew weren’t true.  Both professors I worked under at Stanford published studies that I know weren’t done correctly.  Sadly, they know it too.

This subterfuge can be hard for outsiders to notice.  But sometimes the flaws are things that anybody could be taught to identify.  With just a little bit of guidance, anybody foolish enough to purchase the pop medicine book I worked on would be able to look up the original research papers and read them and realize that they’re garbage.

There’s a catch: most of those papers cost between twenty and thirty dollars a pop.  The chapters I wrote cite nearly a hundred articles.  I’d describe a few studies about the off-label use of this drug, a few about that one, on and on, “so that our readers feel empowered to make their own decisions instead of being held at the paternalistic mercy of their healthcare professionals.”  A noble goal.  But I’m not sure that recommending patients dabble with ineffectual, oft-risky alternative medicines is the best way to pursue it.  Especially when the book publisher was discussing revenue sharing agreements with sellers of some of the weird stuff we shilled.

So, those hundred citations?  You could spend three thousand dollars figuring out that the chapters I wrote are crap.  The situation is slowly getting better – the National Institute of Health has mandated that taxpayer-funded studies be made available after a year, but this doesn’t apply to anything published before 2008, and I’m not sure how keen sick patients will be to twiddle their thumbs for a year before learning the latest information about their diseases.  Plus, there are many granting organizations out there.  Researchers who get their money elsewhere aren’t bound by this requirement.  If somebody asks you, “Would you like to donate money to fight childhood cancer?” and you chip in a buck, you’re actually contributing to the problem.

1403798100_f4ba200fe0_z
Photo by diylibrarian on Flickr.

I was only able to write my chapters of that book because I live next to a big university.  I could stroll to the library and use their permissions to access the papers I’d need.  Sometimes, though, that wasn’t enough.  Each obscure journal, of which there are legion, can cost a university several thousand dollars a year for a subscription.  A few studies I cited were published in specialty journals too narrowly focused for Indiana University to subscribe, so I’d send an email to a buddy still working at Stanford and ask him to send me a copy.

If you get sick and worry yourself into looking for the truth, you’ll probably be out of luck.  Even doing your research at a big state university library might not be enough.

That’s if you keep your research legal.

Or you could search for the papers you need on Sci-Hub.  Then you’d just type the title, complete a CAPTCHA on a page with instructions in Cyrillic (on what was until recently http://www.sci-hub.cc, at least), and, bam!  You have it!  You can spend your thirty dollars on something else.  Food, maybe, or rent.

Of course, this means you are a thief.  The publisher didn’t get the thirty dollars they charge for access to a paper.  And those academic publishers would like for you to feel the same ethical qualms that we’re retraining people to feel when they pirate music or movies.  If you steal, content producers won’t be paid, they’ll starve, and we’ll staunch the flow of beautiful art to which we’ve become accustomed.

The comparison between Napster and Sci-Hub is a false analogy.  Slate correspondent Justin Peters described the perverse economics of academic publishing, in particular the inelastic demand – nobody reads research journals for fun.

With music and movies, purchasing legitimate access funds creators.  Not so in academia.  My laboratory had to pay a journal to publish my thesis work; this is standard practice.  It costs the authors a lot of money to publish a research article, and “content producers” only do it, as opposed to slapping their work up on a personal website for everyone to read free, because they need publication credits on their CVs to keep winning grants.

With music and movies, stealing electronic copies makes content producers sad.  With research articles, it makes them happy.

In fact, almost everyone believes research articles should be free.  At the European Union’s recent Competitiveness Council, the member states agreed that all scientific papers should be freely available by 2020 – these  are the governments whose enforcement is necessary to maintain the current copyright system!  The only people making statements in favor of the status quo are employed by the academic publishers themselves.  Their ideological positions may be swayed somewhat by the $2 billion plus profit margins major publishers are able to extract from their current racket.

Academic publishers would argue that they serve an important role as curators of the myriad discoveries made daily.  This doesn’t persuade me.  The “referees” they rely on to assess whether each study is sound are all unpaid volunteers.  Plus, if the journals were curating well, wouldn’t it have been harder for me to fill that pop medicine book with so much legitimate-looking crap?

Most importantly, by availing yourself of Sci-Hub’s pirated material, you the thief no longer live in ignorance.  With our current healthcare model, ignorance is deadly.  The United States is moving toward an a la carte method of delivering treatment, where sick people are expected to be knowledgeable, price-sensitive consumers rather than patients who place their trust in a physician.  Most sick people no longer have a primary care physician who knows much about their personal lives – instead, doctors are forced for financial reasons to join large corporate conglomerates.  Doctors try their best moment by moment, but they might never see someone a second time.  It’s more important than ever for patients to stay well-informed.

Unless Sci-Hub wins its lawsuit, you probably can’t afford to.

On the low-quality, highly-biased research attempting to dissuade you from wearing a bike helmet.

On the low-quality, highly-biased research attempting to dissuade you from wearing a bike helmet.

Just in case something urgent pops up and you don’t have a chance to finish reading this whole piece, here’s the condensed version: Wear a helmet!

Maybe you’re a great biker.  Maybe you would never, ever crash your bike.  So what?  Do you know how many dudes from my old high school drive around drunk at all times of day?  Doesn’t matter whether or not you’re superfly on two wheels if somebody in a pickup swipes you.

5645160478_7879d153e5_b
Image by waferboard on Flickr.

Conk your head hard, you might die.  Even if you don’t die, a traumatic brain injury can leave your thoughts fuzzy and eyesight blurry for the rest of your life.  Some five million people in the U.S. are suffering from traumatic brain injuries, picked up from concussions during youth sports, exposure to brain-pummeling blasts of sound in the military, sudden impact during car or bike wrecks.  Now they have throbbing headaches, chronic fatigue, troubled sleep, depression …

Your brain makes you you.  And there’s no good way to fix it.  This is tragic, obviously.  If you’re feeling too chipper, go ahead and search Pubmed for the wacky psuedoscience people are trying in their flailing attempts to feel better — and that’s the stuff that’s peer reviewed.  It gets even worse if you search the whole internet.  There’re plenty of quack doctors out there willing to profit off false hope.

It’s not worth risking all that just to keep your hair from getting mussed.

506589238_940610c12c_z
Image by Ruth Ellison on Flickr.

And yet.  Even though everybody knows now that football causes brain injuries, some hundred million people watch the Super Bowl, and over a million high schoolers play.  Even though everybody knows you’re safer wearing a bike helmet, something like 50% of neurosurgeons don’t.

Those doctors know better than anyone else how horrible traumatic brain injuries can be.  But the practice of medicine encourages cognitive dissonance.  Spending all day every day around sick people can wear you down unless you maintain a clear separation between you, the healer, and them, the sick people.  And so, even though their professional organization distributes information saying that everyone should wear a helmet every time they bike, they think of themselves as special.  Separate.

They’re wrong, of course.  Medical doctors get knocked off their bikes just like everybody else.  Indeed, the more training and expense that’s been poured into your brain, the more you ought to value it.  Seems absurd me that anyone would risk easily-preventable brain damage after spending several hundred thousand dollars on an education.

Researchers who don’t wear bike helmets must realize that their cognitive dissonance is absurd.  Because every so often another flawed study is published suggesting that bike helmets make you less safe.  The only explanation I can think of for all these studies is this: some researchers don’t want to wear helmets, and then, because they’re oh-so brilliant, they design & publish crappy experiments to justify their decision.

15792784747_9366f1fe58_b.jpg
Image by Greg Raisman on Flickr.

The first time I learned about these studies was shortly after beginning graduate school.  The New York Times publishes an annual “year in ideas” magazine, and one of the great “ideas” that year was that drivers might think they need to be less cautious when passing a biker with a helmet.

There were obvious problems with this study.  The biggest problem is that only a single biker was studied: the researcher who designed the experiment and who acknowledges that he doesn’t like wearing helmets and wanted to find a negative consequence of helmet-wearing.  This is as far from a double-blinded study as you can get.

Even then, it took bizarre data analysis for him to get the conclusion he wanted.  (Insert obligatory Ronald Coase quotation here: “If you torture the data long enough, it will confess.”)

Any time you obtain lots of data and analyze it lots of different ways, you can make it seem to support the conclusion you want.  You can read a very clear description of this — replete with lots of fun examples — in Gary Smith’s Standard Deviations.

Even though the researcher’s bias probably led him to bicycle more recklessly when he had a helmet on, his data don’t support his conclusion.  Consider this scatterplot prepared by other researchers who re-analyzed his data.

Capture

 

Drivers often don’t pass any more closely when he has a helmet on.  At a glance, the only conclusion I’d make from looking at this is that he was probably safer when he was farther from the curb — there’s less debris farther into the road, and drivers were giving him just as much room.  But, regarding helmets?  The data are a wash.

Of course, given scientific publishing’s & our popular media’s preference for exciting, counter-intuitive results, the correction — you should wear a helmet, after all! — got much less press than the original, incorrect conclusion.

Yet another “maybe you shouldn’t wear a helmet” study was just published.  This one purports to investigate “risk compensation,” the idea that wearing a helmet makes you feel safer, and that the feeling of safety might cause you to bike recklessly, with the net result being that you’re even less safe than before, sans headgear.

It’s a cute idea.  And it’s been studied extensively.  Bikers who usually wear helmets do seem to bike more slowly when they don’t have a helmet on.  That same study found that bikers unaccustomed to helmets didn’t change their behavior when wearing one.  And another study found that helmeted bicyclists typically engage in less risky maneuvers.  (That last study simply tracked people who did & did not choose to wear helmets, as opposed to randomizing the population.  It’s quite possible that people who choose helmets are less likely to take risks in general.)

All of which indicates that, yes, you should wear a helmet.  Don’t worry, kid, you’re not going to psych yourself out and get the itch to careen in front of cars as soon as you strap one on!

But some researchers don’t like these results.  They want to bike without helmets, and they want to feel brilliant while doing so.

Thus, the new study.

This one used a highly contrived scenario, where research subjects were awarded prizes based on how large they could inflate a computer-simulated balloon.  They had to wear headgear, ostensibly to track their eye motion, during the experiment.  For some participants, the eye-tracking cameras were attached to baseball caps, for other participants the cameras were attached to helmets.  Voila!  A blinded study!  These people are now wearing helmets without even realizing it!

The researchers found that helmeted subjects inflated their virtual balloons some 30% larger than baseball-capped subjects.  Voila!  The researchers demonstrated risk compensation!

Except … people don’t act riskier in the real world when they strap on helmets.  Yeah, it’d be somewhat curious if having a helmet on — which clearly won’t protect a digital balloon from disaster — caused people to behave more cavalierly.

Their results hardly show that, however.  They barely even published their results!

sq

Each “virtual balloon” was designed to “pop” after a random number of inflations, between 1 and 128.  Participants “inflated” the balloon by clicking a mouse.  Which seems odd in and of itself — if they’re using balloon imagery, why wouldn’t the thing be more likely to pop when it was big than when it was small?  If you’re claiming that people’s sense of real-world protection from wearing a helmet affects their decisions, you should make your virtual design as close to the real-world as possible.  Otherwise you’re just introducing a strange cognitive disconnect between expectations and results.

What the researchers published was, for each of the 80 study participants, the average number of clicks for all trials in which the virtual balloon did not pop.  I looked through their paper, but I couldn’t find the number of trials excluded because the balloons popped.  So even if they found a significantly lower average number of clicks between helmeted & baseball-capped participants, this doesn’t mean anything.  You’d see this same result if the effect of a helmet was to make people less erratic, i.e. safer.

An erratic participant might choose a number of clicks with high variance — in one trial she might click 30 times, in the next trial 90 times.  A safety-conscious participant might always click the same number of times — always 60 clicks.  Well, if the second balloon for the erratic participant popped, the averages recorded would be 30 clicks for her, 60 clicks for the other subject.  The reckless participant, because of this strange data manipulation, looks more risk averse!

This very well might be what the researchers saw.  With a quick calculation, you can find the optimal strategy for number of clicks… it turns out to be 64 clicks.

bikemath
Here’s that calculation.

Half the time you use this strategy, your balloon will pop, but your total prize will be as high as possible.  And, with the researcher’s strange way of calculating the average number of clicks, someone using this strategy would have an average of 64.  In each trial, they either clicked this number of times, or the balloon popped.

bike helmet graphHere’s the researchers’ graph of their data.  What you can see: zero study subjects wearing caps used the optimal strategy — everyone from that population had an average below 64.  No one from the helmet-wearing group had an average of exactly 64 either, but there’s a cluster of scores nearby.

My conclusion?  Wearing a bike helmet makes people smarter.  The helmet group was much closer to playing the game optimally.

(Um, sure, my conclusion is spurious too.  Most people in both groups played the game the same way.  All that these data show is a small number of players in the helmet group with higher average click numbers — again with the caveat that the method used to calculate the average is bizarre, and wrong.  You can’t really conclude anything from this study.  But you certainly can’t conclude that wearing a helmet influences people to take more risks.)