New research from Baylor College of Medicine in Houston and the City College of New York shows that the visual information you absorb when you see can improve your understanding of the spoken words by as much as sixfold.
Your brain uses the visual information derived from the person's face and lip movements to help you interpret what you hear, and this benefit increases when the sound quality rises to moderately noisy, said Dr. Wei Ji Ma, assistant professor of neuroscience at BCM and the report's lead author, in a report that appears online today in the open access journal PLoS ONE.
"Most people with normal hearing lip-read very well, even though they don't think so," said Ma. "At certain noise levels, lip-reading can increase word recognition performance from 10 to 60 percent correct."
However, when the environment is very noisy or when the voice you are trying to understand is very faint, lip-reading is difficult.
"We find that a minimum sound level is needed for lip-reading to be most effective," said Ma.
This research is the first to study word recognition in a natural setting, where people report freely what they believe is being said. Previous experiments only used limited lists of words for people to choose from.
The lip-reading data help scientists understand how the brain integrates two different kinds of stimuli to come to a conclusion.
Ma and his colleagues constructed a mathematical model that allowed them to predict how successful a person will be at integrating the visual and auditory information.
People actually combine the two stimuli close to optimally, Ma said. What they perceive depends on the reliability of the stimuli.
"Suppose you are a detective," he said. "You have two witnesses to a crime. One is very precise and believable. The other one is not as believable. You take information from both and weigh the believability of each in your determination of what happened."
In a way, lip-reading involves the same kind of integration of information in the brain, he said.
In experiments, videos of individuals were shown in which a person said a word. If the person is presented normally, the visual information provides a great benefit when it is integrated with the auditory information, especially when there is moderate background noise. Surprisingly, if the person is just a "cartoon" that does not truly mouth the word, then the visual information is still helpful, though not as much.
In another study, the person mouths one word but the audio projects another, and often the brain integrates the two stimuli into a totally different perceived word.
"The mathematical model can predict how often the person will understand the word correctly in all these contexts," Ma said.
An example of the visual and audio stimuli used in the experiment can be found at http://bme.engr.ccny.cuny.edu/faculty/parra/bayes-speech/.
Others who took part in this research include Xiang Zhou, Lars A. Ross, John J. Foxe and Lucas C. Parra of The City College of New York in New York City.
When the embargo lifts, the full report can be found at http://dx.plos.org/10.1371/journal.pone.0004638
Glenna Picton | EurekAlert!
Diagnoses: When Are Several Opinions Better Than One?
19.07.2016 | Max-Planck-Institut für Bildungsforschung
High in calories and low in nutrients when adolescents share pictures of food online
07.04.2016 | University of Gothenburg
Terahertz excitation of selected crystal vibrations leads to an effective magnetic field that drives coherent spin motion
Controlling functional properties by light is one of the grand goals in modern condensed matter physics and materials science. A new study now demonstrates how...
Researchers from the Institute for Quantum Computing (IQC) at the University of Waterloo led the development of a new extensible wiring technique capable of controlling superconducting quantum bits, representing a significant step towards to the realization of a scalable quantum computer.
"The quantum socket is a wiring method that uses three-dimensional wires based on spring-loaded pins to address individual qubits," said Jeremy Béjanin, a PhD...
In a paper in Scientific Reports, a research team at Worcester Polytechnic Institute describes a novel light-activated phenomenon that could become the basis for applications as diverse as microscopic robotic grippers and more efficient solar cells.
A research team at Worcester Polytechnic Institute (WPI) has developed a revolutionary, light-activated semiconductor nanocomposite material that can be used...
By forcefully embedding two silicon atoms in a diamond matrix, Sandia researchers have demonstrated for the first time on a single chip all the components needed to create a quantum bridge to link quantum computers together.
"People have already built small quantum computers," says Sandia researcher Ryan Camacho. "Maybe the first useful one won't be a single giant quantum computer...
COMPAMED has become the leading international marketplace for suppliers of medical manufacturing. The trade fair, which takes place every November and is co-located to MEDICA in Dusseldorf, has been steadily growing over the past years and shows that medical technology remains a rapidly growing market.
In 2016, the joint pavilion by the IVAM Microtechnology Network, the Product Market “High-tech for Medical Devices”, will be located in Hall 8a again and will...
14.10.2016 | Event News
14.10.2016 | Event News
12.10.2016 | Event News
24.10.2016 | Earth Sciences
24.10.2016 | Life Sciences
24.10.2016 | Physics and Astronomy