If things keep going like this, it will be as easy to pick up
someone’s thoughts as it is their cell-phone conversation. It’s been
only a few months since I wrote about scientists who had trained a computer to distinguish thoughts:
“Scientists at Carnegie Mellon University showed people drawings of
five tools hammer, drill and the like) and five dwellings (castle,
igloo …) and asked them to think about each object’s properties, uses
and anything else that came to mind. Meanwhile, fMRI measured activity
throughout each volunteer’s brain. As the scientists report this month
in the journal PLoS One, the activity pattern evoked by each object was
so distinctive that the computer could tell with 78 percent accuracy
when someone was thinking about a hammer and not, say, pliers. . . .
Remarkably, the activity patterns—from visual areas to movement area to
regions that encode abstract ideas like the feudal associations of a
castle—were eerily similar from one person to another. 'This
establishes, as never before, that there is a commonality in how
different people’s brains represent the same object,' said CMU’s Tom
Mitchell."
Now the same team, led by computer scientist Mitchell and cognitive
neuroscientist Marcel Just, has taken the next step. As they report in tomorrow’s issue of the journal Science,
they can now identify the unique brain activation patterns associated
with scores of concrete nouns—that is, names for things you can see,
hear, feel, taste or smell.
The team started with the fMRI patterns for 60 concrete nouns,
including words for animals, body parts, buildings, clothing, insects,
vehicles and vegetables. They then had their computer statistically
analyze texts totaling more than 1 trillion words. For each noun, it
calculated how frequently it occurs in the text with any of 25 verbs
that have sensory or movement meanings, such as see, hear, listen, taste, smell, eat, push, drive and lift.
The computer model combined the two sets of data to predict the
activation patterns for thousands of concrete nouns, and racked up an
accuracy of 77 percent.
“We believe we have identified a number of the basic building blocks
that the brain uses to represent meaning,” said Mitchell in a
statement. “Coupled with computational methods that capture the meaning
of a word by how it is used in text files, these building blocks can be
assembled to predict neural activation patterns for any concrete noun.”
A big reason this seems to work is that, as Just explained, people
“are fundamentally perceivers and actors. So the brain represents the
meaning of a concrete noun in areas associated with how people sense it
or manipulate it. The meaning of an apple, for instance, is represented
in brain areas responsible for tasting, for smelling, for chewing. An
apple is what you do with it.”
Besides sensory and motor areas, brain regions that become active
when people think of concrete nouns include frontal areas, which plan
and also encode long-term memory. Thoughts of apple, it seems, triggers a remembrance of apples past, and also of thoughts about how to get one.
That’s why the model struggles to distinguish apple from pear, for instance, and it also suggests that words other than concrete nouns—by, under, over, around, soon, later, enormous
and every other preposition, adjective and adverb, not to mention
verbs—will be tough for the computer to read from patterns of brain
activity. Yet these words are, needless to say, crucial to determining
meaning. The computer can’t tell It’s time to smuggle the bomb into the harbor from It’s criminal how poorly the harbor is defended against bombs. But at the rate scientists are going, it might not be long before it can.