The three-year project, which starts next month, will collect data for lip-reading and use it to create machines that automatically convert videos of lip-motions into text.
It builds on work already carried out at UEA to develop state-of-the-art speech reading systems.
The university is teaming up with the Centre for Vision, Speech & Signal Processing at Surrey University, who have built accurate and reliable face and lip trackers, and the Home Office Scientific Development Branch, who want to investigate the feasibility of using the technology for crime fighting.
The team also hope to carry out computerised lip-reading of other languages.
While it is known that humans can and do lip-read, not much is known about exactly what visual information is needed for effective lip-reading. Human lip-reading can be unreliable, even using trained lip-readers.
Dr Richard Harvey, senior lecturer at UEA’s School of Computing Sciences, is leading the project, which has been awarded £391,814 by the Engineering and Physical Sciences Research Council.
“It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill.”
Dr Harvey added: “The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence.”
As well as crime fighting there could be other potential uses for the technology, such as installing a camera in a mobile phone, or on the dash board for in-car speech recognition systems.
Another reason for developing computerised lip-reading is that the number of trained lip-readers is falling, mainly because people tend to be taught to sign instead.
Dr Harvey said: “To be effective the systems must accurately track the head over a variety of poses, extract numbers, or features, that describe the lips and then learn what features correspond to what text.
“To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognise visual speech.
“The work will be highly experimental. We hope to have produced a system that will demonstrate the ability to lip-read in more general situations than we have done so far.”
Press Office | alfa
Drones can almost see in the dark
20.09.2017 | Universität Zürich
World first: 'Storing lightning inside thunder'
18.09.2017 | University of Sydney
Whispering gallery mode (WGM) resonators are used to make tiny micro-lasers, sensors, switches, routers and other devices. These tiny structures rely on a...
Using ultrafast flashes of laser and x-ray radiation, scientists at the Max Planck Institute of Quantum Optics (Garching, Germany) took snapshots of the briefest electron motion inside a solid material to date. The electron motion lasted only 750 billionths of the billionth of a second before it fainted, setting a new record of human capability to capture ultrafast processes inside solids!
When x-rays shine onto solid materials or large molecules, an electron is pushed away from its original place near the nucleus of the atom, leaving a hole...
For the first time, physicists have successfully imaged spiral magnetic ordering in a multiferroic material. These materials are considered highly promising candidates for future data storage media. The researchers were able to prove their findings using unique quantum sensors that were developed at Basel University and that can analyze electromagnetic fields on the nanometer scale. The results – obtained by scientists from the University of Basel’s Department of Physics, the Swiss Nanoscience Institute, the University of Montpellier and several laboratories from University Paris-Saclay – were recently published in the journal Nature.
Multiferroics are materials that simultaneously react to electric and magnetic fields. These two properties are rarely found together, and their combined...
MBM ScienceBridge GmbH successfully negotiated a license agreement between University Medical Center Göttingen (UMG) and the biotech company Tissue Systems Holding GmbH about commercial use of a multi-well tissue plate for automated and reliable tissue engineering & drug testing.
MBM ScienceBridge GmbH successfully negotiated a license agreement between University Medical Center Göttingen (UMG) and the biotech company Tissue Systems...
Pathogenic bacteria are becoming resistant to common antibiotics to an ever increasing degree. One of the most difficult germs is Pseudomonas aeruginosa, a...
19.09.2017 | Event News
12.09.2017 | Event News
06.09.2017 | Event News
20.09.2017 | Life Sciences
20.09.2017 | Power and Electrical Engineering
20.09.2017 | Physics and Astronomy