The three-year project, which starts next month, will collect data for lip-reading and use it to create machines that automatically convert videos of lip-motions into text.
It builds on work already carried out at UEA to develop state-of-the-art speech reading systems.
The university is teaming up with the Centre for Vision, Speech & Signal Processing at Surrey University, who have built accurate and reliable face and lip trackers, and the Home Office Scientific Development Branch, who want to investigate the feasibility of using the technology for crime fighting.
The team also hope to carry out computerised lip-reading of other languages.
While it is known that humans can and do lip-read, not much is known about exactly what visual information is needed for effective lip-reading. Human lip-reading can be unreliable, even using trained lip-readers.
Dr Richard Harvey, senior lecturer at UEA’s School of Computing Sciences, is leading the project, which has been awarded £391,814 by the Engineering and Physical Sciences Research Council.
“It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill.”
Dr Harvey added: “The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence.”
As well as crime fighting there could be other potential uses for the technology, such as installing a camera in a mobile phone, or on the dash board for in-car speech recognition systems.
Another reason for developing computerised lip-reading is that the number of trained lip-readers is falling, mainly because people tend to be taught to sign instead.
Dr Harvey said: “To be effective the systems must accurately track the head over a variety of poses, extract numbers, or features, that describe the lips and then learn what features correspond to what text.
“To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognise visual speech.
“The work will be highly experimental. We hope to have produced a system that will demonstrate the ability to lip-read in more general situations than we have done so far.”
Press Office | alfa
Information integration and artificial intelligence for better diagnosis and therapy decisions
24.05.2017 | Fraunhofer MEVIS - Institut für Bildgestützte Medizin
World's thinnest hologram paves path to new 3-D world
18.05.2017 | RMIT University
Scientists have developed a new method of characterizing graphene’s properties without applying disruptive electrical contacts, allowing them to investigate both the resistance and quantum capacitance of graphene and other two-dimensional materials. Researchers from the Swiss Nanoscience Institute and the University of Basel’s Department of Physics reported their findings in the journal Physical Review Applied.
Graphene consists of a single layer of carbon atoms. It is transparent, harder than diamond and stronger than steel, yet flexible, and a significantly better...
The world's highest gain high power laser amplifier - by many orders of magnitude - has been developed in research led at the University of Strathclyde.
The researchers demonstrated the feasibility of using plasma to amplify short laser pulses of picojoule-level energy up to 100 millijoules, which is a 'gain'...
Staphylococcus aureus is a feared pathogen (MRSA, multi-resistant S. aureus) due to frequent resistances against many antibiotics, especially in hospital infections. Researchers at the Paul-Ehrlich-Institut have identified immunological processes that prevent a successful immune response directed against the pathogenic agent. The delivery of bacterial proteins with RNA adjuvant or messenger RNA (mRNA) into immune cells allows the re-direction of the immune response towards an active defense against S. aureus. This could be of significant importance for the development of an effective vaccine. PLOS Pathogens has published these research results online on 25 May 2017.
Staphylococcus aureus (S. aureus) is a bacterium that colonizes by far more than half of the skin and the mucosa of adults, usually without causing infections....
Physicists from the University of Würzburg are capable of generating identical looking single light particles at the push of a button. Two new studies now demonstrate the potential this method holds.
The quantum computer has fuelled the imagination of scientists for decades: It is based on fundamentally different phenomena than a conventional computer....
An international team of physicists has monitored the scattering behaviour of electrons in a non-conducting material in real-time. Their insights could be beneficial for radiotherapy.
We can refer to electrons in non-conducting materials as ‘sluggish’. Typically, they remain fixed in a location, deep inside an atomic composite. It is hence...
24.05.2017 | Event News
23.05.2017 | Event News
22.05.2017 | Event News
30.05.2017 | Life Sciences
30.05.2017 | Power and Electrical Engineering
29.05.2017 | Earth Sciences