Speaking and listening – interbrain communication
So I’ve been writing a lot of ‘these things bug me’ posts lately. I figured I’d put up something cool today, and save my latest rant for another post.
One of my profs sent me a cool new article on speech imaging. It’s a study conducted at Princeton, by Stephens et al.
One of the big topics in speech research is communication. I know this seems obvious, but there are a lot of studies devoted to how people perceive speech – how phonemes make words, how words have meaning, how words fit together to make a grammatical sentence; and to production, with how people pick appropriate words, pronounce the phonemes, construct grammatical sentences, and convey meaning through prosody. Already I’ve left a lot out, and made a bunch of assumptions about hotly debated topics.
Now, my lab is big on the motor theory of speech. That’s a topic for another day, but suffice to say we are very interested in how people go from perceiving to producing. Because speech is not static. We hear things and need to be able to respond very quickly, almost at the rate of unconscious action. We only speak as though we were picking words out of a lexicon and parsing them syntactically when we’re speaking an unfamiliar language, have some injury or disorder, or are in a debilitated condition. That last is why I’m holding off on a delicious whisky sour until after I’m done writing this post.
There are a bunch of ideas explaining in different way how we are able to communicate so effectively, seamlessly picking up our side of a conversation in real time, and following along with a spoken narrative even under noisy conditions or confusing accents.
Stephens et al recorded functional MRI from a participant who related an unrehearsed story in English, about an experience during a high school prom. They also recorded a similarly unrehearsed personal story from a native Russian speaker, also while recording functional MRI. This was done a few times to acquaint the participant with the procedure, and then a single story for each speaker was picked.
MRI machines make a lot of noise, so they used a clever technique with two microphones that allowed them to remove the scanner noise from the story in post-processing, without making the speech sound artificial. I expect we’ll be seeing this technique more often, as it’s useful to be able to record verbal responses.
After all that, they played the story to a bunch of listeners, and recorded their brain activity using fMRI while the listeners listened to the stories in both Russian and English.
Now, here’s the cool part. Using a moving window of -4 to -1.5 to 0 to 1.5 to 4 seconds, they compared brain activity in the speaker to brain activity in the listeners during the entire length of the story. Afterwards, the listeners were tested on their recall and comprehension.
There was, excitingly, a large overlap between active regions in the speaker and the listeners. This is cool because a number of theories of speech postulate that the same regions involved in producing speech are also involved in perceiving it. Further, they found that much of this activation in the listeners lagged behind the speaker by a few seconds, implying that there was a causal relationship – that the regions active in the speaker were, via the speech, causing the same regions to become active in the listeners. In this way, a spoken narrative can be thought of as a method for one brain to induce specific activity in another (see my previous post on us being our brains).
But even cooler, was a few regions which in the listeners were active a few seconds ahead of the speakers. There were mostly frontal regions involved in planning and other executive functions. The authors suggest this is due to anticipation on the part of the listeners, a sort of heuristic predictive action to facilitate comprehension. Indeed, those listeners which had more activity in advance of the speaker’s showed improved comprehension and recall after the experiment.
Furthermore, this was all dependent on actual understanding of the story. The only speaker-listener coupling during the Russian story was in the primary auditory regions, implying that while the listener’s brains were being activated by the sounds being heard, there was no further processing specific to the story.
Even more fun than that, when they asked the speaker to relate a new story while being scanned, and compared their brain activity patterns to this listeners from the original story (to see if there is simply a regular spatial-temporal pattern of activity while relating and listening to a narrative), they found very little activity coupling. It turns out that while the generalities of activation are there, actual activity coupling only seems to occur during shared communication.
I really look forward to seeing these results replicated. The actual analysis sounds a little tricky, but the experiment itself is simple and elegant, and if the results bear out it has some interesting implications for the study of coupled speech perception and production.
