Note: the original version of this post included a section on individual teacher ratings. I could not find public references for the statements I made, and have deleted these two paragraphs. My apologies!

I recently placed a copy of How to Lie with Statistics in a little free library near campus.  Not because I want people to be more deceitful – if you don’t understand how to trick others, then you yourself will be easy game.  Numbers sound like facts.  They can be used for malicious ends.

Consider medical ratings.  These are ostensibly beneficial – prospective patients get to learn how well-trained their doctors are! 

Saurabh Jha wrote an excellent essay explaining why these rankings are misleading, “When a Bad Surgeon Is the One You Want.”  In brief, doctors who take easy cases will improve their ratings – their patients are more likely to have good outcomes.  When doctors are assessed on their patients’ outcomes, then the doctors who take hard cases will appear to be incompetent.  Even if they are much better at their craft than others.

The same phenomenon holds in teaching. Schools and teachers are often evaluated based on their students’ performance, without normalizing for the unique challenges faced by different populations of kids. 

This week, the Indiana Department of Education released federal evaluations of local schools. 

The elementary school located amidst our town’s most expensive houses, at which the lowest percentage of students receive free or reduced-price lunch, was rated as “exceeding expectations.”  

The elementary schools that serve our town’s most disadvantaged students – one of which holds bilingual classes in English and American Sign Language to support deaf children, and has 86% of students receiving free or reduced-priced lunch – were rated as not meeting expectations.

My spouse and I are sending our own children to one of the schools that was rated as not meeting expectations.  We know a fair bit about education – among other things, my spouse is the editor-in-chief of a national journal of teacher writing.  I’ve observed classrooms in this low-rated school, and they are excellent.

But teacher morale is low, because the teachers are continually evaluated as being sub-par, despite the fact that they have chosen to work harder than others.  Our school district is mandating that teachers in the low-rated schools spend time on unfulfilling test-prep regimes, even though these practices are known to further alienate under-resourced students.

Our nation’s school administrators ought to read How to Lie with Statistics, it seems.  They’ve looked at a set of numbers and allowed themselves to be misled.  Which bodes ill for the learners in their care.