Princeton researchers have for the first time matched images of brain activity with categories of words related to the concepts a person is thinking about.

The research may lead to a better understanding of how people consider meaning and context when reading or thinking.

The researchers worked from fMRI images of brain activity. For those scans, nine people were presented with the word and picture of five concrete objects from 12 categories.

The drawing and word for the 60 total objects were displayed in random order until each had been shown six times. Each time an image and word appeared, participants were asked to visualize the object and its properties for three seconds as the fMRI scanner recorded their brain activity.

Separately, the researchers constructed a list of topics with which to categorize the fMRI data. They used a computer program to condense 3,500 articles about concrete objects from the online encyclopedia Wikipedia into all the topics the articles covered. The articles included a broad array of subjects, such as an airplane, heroin, birds and manual transmission. The program came up with 40 possible topics — such as aviation, drugs, animals or machinery — with which the articles could relate. Each topic was defined by the words most associated with it.

The computer then created a database of topics and associated words that were free from the researchers’ biases. The topic database let the researchers objectively arrange the fMRI images by subject matter. To do so, the team searched the brain scans of related objects for similar activity to determine common brain patterns for an entire subject.

The neural response for thinking about “furniture,” for example, was determined by the common patterns found in the fMRI images for “table,” “chair,” “bed,” “desk,” and “dresser.” At the same time, the team established all the words associated with “furniture” by matching each fMRI image with related words from the Wikipedia-based list.

Based on the similar brain activity and related words, the researchers concluded that the same neural response would appear whenever a person thought of any of the words related to furniture. And a scientist analyzing that brain activity would know that person was thinking of furniture. The same would follow for any topic.

The researchers found that they could confidently determine from an fMRI image the general topic on a participant’s mind, but that deciphering specific objects was trickier. For example, they could compare the fMRI scan for “carrot” against that for “cow” and safely say that at the time the participant had thought about vegetables in the first example instead of animals. In turn, they could say that the person most likely thought of other words related to vegetables, as opposed to words related to animals.

On the other hand, when the scan for “carrot” was compared to that for “celery,” the researchers knew the participant had thought of vegetables, but they could not identify related words unique to either object.

Ref.: Francisco Pereira, Greg Detre, and Matthew Botvinick, Generating text from functional brain images, Frontiers in Human Neuroscience, 2011; [DOI: 10.3389/fnhum.2011.00072]