In Philosophical Investigations (translated by G. E. M. Anscombe), Ludwig Wittgenstein argues that something strange occurs when we learn a language. As an example, he cites the problems that could arise when you point at something and describe what you see:
The definition of the number two, “That is called ‘two’ “ – pointing to two nuts – is perfectly exact. But how can two be defined like that? The person one gives the definition to doesn’t know what one wants to call “two”; he will suppose that “two” is the name given to this group of nuts!
I laughed aloud when I read this statement. I borrowed Philosophical Investigations a few months after the birth of our second child, and I had spent most of his first day pointing at various objects in the hospital maternity ward and saying to him, “This is red.” “This is red.”
“This is red.”
Of course, the little guy didn’t understand language yet, so he probably just thought, the warm carry-me object is babbling again.
Over time, though, this is how humans learn. Wittgenstein’s mistake here is to compress the experience of learning a language into a single interaction (philosophers have a bad habit of forgetting about the passage of time – a similar fallacy explains Zeno’s paradox). Instead of pointing only at two nuts, a parent will point to two blocks – “This is two!” and two pillows – “See the pillows? There are two!” – and so on.
As a child begins to speak, it becomes even easier to learn – the kid can ask “Is this two?”, which is an incredibly powerful tool for people sufficiently comfortable making mistakes that they can dodge confirmation bias.
(When we read the children’s story “In a Dark Dark Room,” I tried to add levity to the ending by making a silly blulululu sound to accompany the ghost, shown to the left of the door on this cover. Then our youngest began pointing to other ghost-like things and asking, “blulululu?” Is that skeleton a ghost? What about this possum?)
When people first programmed computers, they provided definitions for everything. A ghost is an object with a rounded head that has a face and looks very pale. This was a very arduous process – my definition of a ghost, for instance, is leaving out a lot of important features. A rigorous definition might require pages of text.
Now, programmers are letting computers learn the same way we do. To teach a computer about ghosts, we provide it with many pictures and say, “Each of these pictures has a ghost.” Just like a child, the computer decides for itself what features qualify something for ghost-hood.
In the beginning, this process was inscrutable. A trained algorithm could say “This is a ghost!”, but it couldn’t explain why it thought so.
From Philosophical Investigations:
And what does ‘pointing to the shape’, ‘pointing to the color’ consist in? Point to a piece of paper. – And now point to its shape – now to its color – now to its number (that sounds queer). – How did you do it? – You will say that you ‘meant’ a different thing each time you pointed. And if I ask how that is done, you will say you concentrated your attention on the color, the shape, etc. But I ask again: how is that done?
After this passage, Wittgenstein speculates on what might be going through a person’s head when pointing at different features of an object. A team at Google working on automated image analysis asked the same question of their algorithm, and made an output for the algorithm to show what it did when it “concentrated its attention.”
Here’s a beautiful image from a recent New York Times article about the project, “Google Researchers Are Learning How Machines Learn.” When the algorithm is specifically instructed to “point to its shape,” it generates a bizarre image of an upward-facing fish flanked by human eyes (shown bottom center, just below the purple rectangle). That is what the algorithm is thinking of when it “concentrates its attention” on the vase’s shape.
At this point, we humans could quibble. We might disagree that the fish face really represents the platonic ideal of a vase. But at least we know what the algorithm is basing its decision on.
Usually, that’s not the case. After all, it took a lot of work for Google’s team to make their algorithm spit out images showing what it was thinking about. With most self-trained neural networks, we know only its success rate – even the designers will have no idea why or how it works.
Which can lead to some stunningly bizarre failures.
It’s possible to create images that most humans recognize as one thing, and that an image-analysis algorithm recognizes as something else. This is a rather scary opportunity for terrorism in a world of self-driving cars; street signs could be defaced in such a way that most human onlookers would find the graffiti unremarkable, but an autonomous car would interpret in a totally new way.
In the world of criminal justice, inscrutable algorithms are already used to determine where police officers should patrol. The initial hope was that this system would be less biased – except that the algorithm was trained on data that came from years of racially-motivated enforcement. Minorities are still more likely to be apprehended for equivalent infractions.
And a new artificial intelligence algorithm could be used to determine whether a crime was “gang related.” The consequences of error can be terrible, here: in California, prisoners could be shunted to solitary for decades if they were suspected of gang affiliation. Ambiguous photographs on somebody’s social media site were enough to subject a person to decades of torture.
When an algorithm thinks that the shape of a vase is a fish flanked by human eyes, it’s funny. But it’s a little less comedic when an algorithm’s mistake ruins somebody’s life – if an incident is designated as a “gang-related crime”, prison sentences can be egregiously long, or send someone to solitary for long enough to cause “anxiety, depression, and hallucinations until their personality is completely destroyed.”
Here’s a poem I received in the mail recently:
LOCKDOWN
by Pouncho
For 30 days and 30 nights
I stare at four walls with hate written
over them.
Falling to my knees from the body blows
of words.
It damages the mind.
I haven’t had no sleep.
How can you stop mental blows, torture,
and names –
They spread.
I just wanted to scream:
Why?
For 30 days and 30 nights
My mind was in isolation.
In The Pearl, John Steinbeck reveals how this glut of information can cause us to be hoodwinked. A poor diver becomes suddenly wealthy when he finds a giant pearl. The diver’s infant child was stung by a scorpion and has begun to recover … but a greedy doctor would rather the child receive an expensive cure. The doctor knows that he can fool the diver by drawing his attention to details that never seemed important before.
Cancer cells continue growing without regard for the neighbors they’re crowding. They carry on dividing – spewing forth copies of themselves – long after a team-player would’ve snuffed itself.
Frank Brown Cloud 