Do you hear what i see?

This study confirms a number of recent findings but contradicts classical thinking, in which hearing, taste, touch, sight, and smell are each processed in distinct areas of the brain and only later integrated. The new research, led by Christoph Kayser, PhD, at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany, was published in the February 21 issue of The Journal of Neuroscience.

“This study confirms that what we used to call the 'auditory cortex' should really be thought of as much more complex in terms of its response properties,” says Robert Zatorre, PhD, head of the auditory cognitive neuroscience laboratory at McGill University. “The textbook-standard view of sensory systems as isolated from one another is no longer tenable.” Zatorre did not participate in the study.

Kayser's team used functional magnetic resonance imaging to draw a diagram of 11 small, tightly packed fields in the monkeys' auditory cortex. Each field has a separate map that covers the full range of frequencies. Scans recorded activity in the monkeys' brains while they watched a video, with and without sound, and listened separately to the accompanying sound. The researchers found that fields in the hindmost part of the auditory cortex showed activity when the monkeys watched the video without sound, and activity was enhanced when the video was presented simultaneously with the sound.

“This finding suggests that sensory integration, which is so fundamental to complex mental activity, takes place at very early processing stages,” says Daniel Tranel, PhD, of the University of Iowa, who is not affiliated with the study. “This knowledge could help scientists pinpoint sources of extraordinary sensory processing, such as creativity and genius, as well as abnormal sensory processing, as seen in schizophrenia.”

Kayser notes that the findings also could be used to reveal the role of audio-visual integration in communication or to help pin down where sounds are coming from. “Clearly, our acoustical understanding often improves if we can see the lips of the speaker — for example at a crowded cocktail party,” he says. “However, currently it is not clear whether and how audio-visual interactions are specialized for the processing of communication signals. “The present study clearly shows where in the auditory system researchers have to focus.”

The work was supported by the Max Planck Society, German Research Foundation, and Alexander von Humboldt Foundation.

The Journal of Neuroscience is published by the Society for Neuroscience, an organization of more than 36,500 basic scientists and clinicians who study the brain and nervous system. Kayser can be reached at christoph.kayser@tuebingen.mpg.de.