I recently placed a copy of How to Lie with Statistics in a little free library near campus. Not because I want people to be more deceitful – if you don’t understand how to trick others, then you yourself will be easy game. Numbers sound like facts. They can be used for malicious ends.
Consider medical ratings. These are ostensibly beneficial – prospective patients get to learn how well-trained their doctors are!
Saurabh Jha wrote an excellent essay explaining why these rankings are misleading, “When a Bad Surgeon Is the One You Want.” In brief, doctors who take easy cases will improve their ratings – their patients are more likely to have good outcomes. When doctors are assessed on their patients’ outcomes, then the doctors who take hard cases will appear to be incompetent. Even if they are much better at their craft than others.
The same phenomenon holds in teaching.
In our school district, teachers receive a salary bonus if they are reviewed as “highly effective.” My spouse has never received this bonus. She was recognized as being the best early-career biology teacher in the country; for multiple years, one of the half-dozen best teachers in our state; worth inviting to address graduates at Stanford’s School of Education. But within our school district, she is considered a mediocre teacher.
The reason? Teachers are evaluated based on their students’ performance, and my spouse insists that half her teaching schedule be devoted to high-need students. These students don’t score as well on tests, which is considered evidence that anyone who works with them is a low-quality teacher.
This week, the Indiana Department of Education released federal evaluations of local schools.
The elementary school located amidst our town’s most expensive houses, at which the lowest percentage of students receive free or reduced-price lunch, was rated as “exceeding expectations.”
The elementary schools that serve our town’s most disadvantaged students – one of which holds bilingual classes in English and American Sign Language to support deaf children, and has 86% of students receiving free or reduced-priced lunch – were rated as not meeting expectations.
My spouse and I are sending our own children to one of the schools that was rated as not meeting expectations. We know a fair bit about education – among other things, my spouse is the editor-in-chief of a national journal of teacher writing. I’ve observed classrooms in this low-rated school, and they are excellent.
But teacher morale is low, because the teachers are continually evaluated as being sub-par, despite the fact that they have chosen to work harder than others. Our school district is mandating that teachers in the low-rated schools waste time on unfulfilling test-prep regimes, even though these practices are known to further alienate under-resourced students.
Our nation’s school administrators ought to read How to Lie with Statistics, it seems. They’ve looked at a set of numbers and allowed themselves to be misled. Which bodes ill for the learners in their care.