The three-year project, which starts next month, will collect data for lip-reading and use it to create machines that automatically convert videos of lip-motions into text.
It builds on work already carried out at UEA to develop state-of-the-art speech reading systems.
The university is teaming up with the Centre for Vision, Speech & Signal Processing at Surrey University, who have built accurate and reliable face and lip trackers, and the Home Office Scientific Development Branch, who want to investigate the feasibility of using the technology for crime fighting.
The team also hope to carry out computerised lip-reading of other languages.
While it is known that humans can and do lip-read, not much is known about exactly what visual information is needed for effective lip-reading. Human lip-reading can be unreliable, even using trained lip-readers.
Dr Richard Harvey, senior lecturer at UEA’s School of Computing Sciences, is leading the project, which has been awarded £391,814 by the Engineering and Physical Sciences Research Council.
“It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill.”
Dr Harvey added: “The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence.”
As well as crime fighting there could be other potential uses for the technology, such as installing a camera in a mobile phone, or on the dash board for in-car speech recognition systems.
Another reason for developing computerised lip-reading is that the number of trained lip-readers is falling, mainly because people tend to be taught to sign instead.
Dr Harvey said: “To be effective the systems must accurately track the head over a variety of poses, extract numbers, or features, that describe the lips and then learn what features correspond to what text.
“To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognise visual speech.
“The work will be highly experimental. We hope to have produced a system that will demonstrate the ability to lip-read in more general situations than we have done so far.”
Press Office | alfa
Snake-inspired robot uses kirigami to move
22.02.2018 | Harvard John A. Paulson School of Engineering and Applied Sciences
Camera technology in vehicles: Low-latency image data compression
22.02.2018 | Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI
Quantum computers may one day solve algorithmic problems which even the biggest supercomputers today can’t manage. But how do you test a quantum computer to...
For the first time, a team of researchers at the Max-Planck Institute (MPI) for Polymer Research in Mainz, Germany, has succeeded in making an integrated circuit (IC) from just a monolayer of a semiconducting polymer via a bottom-up, self-assembly approach.
In the self-assembly process, the semiconducting polymer arranges itself into an ordered monolayer in a transistor. The transistors are binary switches used...
Breakthrough provides a new concept of the design of molecular motors, sensors and electricity generators at nanoscale
Researchers from the Institute of Organic Chemistry and Biochemistry of the CAS (IOCB Prague), Institute of Physics of the CAS (IP CAS) and Palacký University...
For photographers and scientists, lenses are lifesavers. They reflect and refract light, making possible the imaging systems that drive discovery through the microscope and preserve history through cameras.
But today's glass-based lenses are bulky and resist miniaturization. Next-generation technologies, such as ultrathin cameras or tiny microscopes, require...
Scientists from the University of Zurich have succeeded for the first time in tracking individual stem cells and their neuronal progeny over months within the intact adult brain. This study sheds light on how new neurons are produced throughout life.
The generation of new nerve cells was once thought to taper off at the end of embryonic development. However, recent research has shown that the adult brain...
15.02.2018 | Event News
13.02.2018 | Event News
12.02.2018 | Event News
22.02.2018 | Life Sciences
22.02.2018 | Physics and Astronomy
22.02.2018 | Earth Sciences