Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Smartphones might soon develop emotional intelligence

04.12.2012
An algorithm for speech-based emotion classification

If you think having your phone identify the nearest bus stop is cool, wait until it identifies your mood.

New research by a team of engineers at the University of Rochester may soon make that possible. At the IEEE Workshop on Spoken Language Technology on Dec. 5, the researchers will describe a new computer program that gauges human feelings through speech, with substantially greater accuracy than existing approaches.

Surprisingly, the program doesn't look at the meaning of the words. "We actually used recordings of actors reading out the date of the month – it really doesn't matter what they say, it's how they're saying it that we're interested in," said Wendi Heinzelman, professor of electrical and computer engineering.

Heinzelman explained that the program analyzes 12 features of speech, such as pitch and volume, to identify one of six emotions from a sound recording. And it achieves 81 percent accuracy – a significant improvement on earlier studies that achieved only about 55 percent accuracy.

The research has already been used to develop a prototype of an app. The app displays either a happy or sad face after it records and analyzes the user's voice. It was built by one of Heinzelman's graduate students, Na Yang, during a summer internship at Microsoft Research. "The research is still in its early days," Heinzelman added, "but it is easy to envision a more complex app that could use this technology for everything from adjusting the colors displayed on your mobile to playing music fitting to how you're feeling after recording your voice."

Heinzelman and her team are collaborating with Rochester psychologists Melissa Sturge-Apple and Patrick Davies, who are currently studying the interactions between teenagers and their parents. "A reliable way of categorizing emotions could be very useful in our research,". Sturge-Apple said. "It would mean that a researcher doesn't have to listen to the conversations and manually input the emotion of different people at different stages."

Teaching a computer to understand emotions begins with recognizing how humans do so.

"You might hear someone speak and think 'oh, he sounds angry!' But what is it that makes you think that?" asks Sturge-Apple. She explained that emotion affects the way people speak by altering the volume, pitch and even the harmonics of their speech. "We don't pay attention to these features individually, we have just come to learn what angry sounds like – particularly for people we know," she adds.

But for a computer to categorize emotion it needs to work with measurable quantities. So the researchers established 12 specific features in speech that were measured in each recording at short intervals. The researchers then categorized each of the recordings and used them to teach the computer program what "sad," "happy," "fearful," "disgusted," or "neutral" sound like.

The system then analyzed new recordings and tried to determine whether the voice in the recording portrayed any of the known emotions. If the computer program was unable to decide between two or more emotions, it just left that recording unclassified.

"We want to be confident that when the computer thinks the recorded speech reflects a particular emotion, it is very likely it is indeed portraying this emotion," Heinzelman explained.

Previous research has shown that emotion classification systems are highly speaker dependent; they work much better if the system is trained by the same voice it will analyze. "This is not ideal for a situation where you want to be able to just run an experiment on a group of people talking and interacting, like the parents and teenagers we work with," Sturge-Apple explained.

Their new results also confirm this finding. If the speech-based emotion classification is used on a voice different from the one that trained the system, the accuracy dropped from 81 percent to about 30 percent. The researchers are now looking at ways of minimizing this effect, for example, by training the system with a voice in the same age group and of the same gender. As Heinzelman said, "there are still challenges to be resolved if we want to use this system in an environment resembling a real-life situation, but we do know that the algorithm we developed is more effective than previous attempts."

Na Yang was awarded a grant by the International Speech Communication Association to attend the SLT Workshop.

For more information on the project visit http://www.ece.rochester.edu/projects/wcng/project_bridge.html.

About the University of Rochester

The University of Rochester is one of the nation's leading private universities. Located in Rochester, N.Y., the University gives students exceptional opportunities for interdisciplinary study and close collaboration with faculty through its unique cluster-based curriculum. Its College, School of Arts and Sciences, and Hajim School of Engineering and Applied Sciences are complemented by its Eastman School of Music, Simon School of Business, Warner School of Education, Laboratory for Laser Energetics, School of Medicine and Dentistry, School of Nursing, Eastman Institute for Oral Health, and the Memorial Art Gallery.

Leonor Sierra | EurekAlert!
Further information:
http://www.rochester.edu

More articles from Information Technology:

nachricht New technology enables 5-D imaging in live animals, humans
16.01.2017 | University of Southern California

nachricht Fraunhofer FIT announces CloudTeams collaborative software development platform – join it for free
10.01.2017 | Fraunhofer-Institut für Angewandte Informationstechnik FIT

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Designing Architecture with Solar Building Envelopes

Among the general public, solar thermal energy is currently associated with dark blue, rectangular collectors on building roofs. Technologies are needed for aesthetically high quality architecture which offer the architect more room for manoeuvre when it comes to low- and plus-energy buildings. With the “ArKol” project, researchers at Fraunhofer ISE together with partners are currently developing two façade collectors for solar thermal energy generation, which permit a high degree of design flexibility: a strip collector for opaque façade sections and a solar thermal blind for transparent sections. The current state of the two developments will be presented at the BAU 2017 trade fair.

As part of the “ArKol – development of architecturally highly integrated façade collectors with heat pipes” project, Fraunhofer ISE together with its partners...

Im Focus: How to inflate a hardened concrete shell with a weight of 80 t

At TU Wien, an alternative for resource intensive formwork for the construction of concrete domes was developed. It is now used in a test dome for the Austrian Federal Railways Infrastructure (ÖBB Infrastruktur).

Concrete shells are efficient structures, but not very resource efficient. The formwork for the construction of concrete domes alone requires a high amount of...

Im Focus: Bacterial Pac Man molecule snaps at sugar

Many pathogens use certain sugar compounds from their host to help conceal themselves against the immune system. Scientists at the University of Bonn have now, in cooperation with researchers at the University of York in the United Kingdom, analyzed the dynamics of a bacterial molecule that is involved in this process. They demonstrate that the protein grabs onto the sugar molecule with a Pac Man-like chewing motion and holds it until it can be used. Their results could help design therapeutics that could make the protein poorer at grabbing and holding and hence compromise the pathogen in the host. The study has now been published in “Biophysical Journal”.

The cells of the mouth, nose and intestinal mucosa produce large quantities of a chemical called sialic acid. Many bacteria possess a special transport system...

Im Focus: Newly proposed reference datasets improve weather satellite data quality

UMD, NOAA collaboration demonstrates suitability of in-orbit datasets for weather satellite calibration

"Traffic and weather, together on the hour!" blasts your local radio station, while your smartphone knows the weather halfway across the world. A network of...

Im Focus: Repairing defects in fiber-reinforced plastics more efficiently

Fiber-reinforced plastics (FRP) are frequently used in the aeronautic and automobile industry. However, the repair of workpieces made of these composite materials is often less profitable than exchanging the part. In order to increase the lifetime of FRP parts and to make them more eco-efficient, the Laser Zentrum Hannover e.V. (LZH) and the Apodius GmbH want to combine a new measuring device for fiber layer orientation with an innovative laser-based repair process.

Defects in FRP pieces may be production or operation-related. Whether or not repair is cost-effective depends on the geometry of the defective area, the tools...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

12V, 48V, high-voltage – trends in E/E automotive architecture

10.01.2017 | Event News

2nd Conference on Non-Textual Information on 10 and 11 May 2017 in Hannover

09.01.2017 | Event News

Nothing will happen without batteries making it happen!

05.01.2017 | Event News

 
Latest News

Multiregional brain on a chip

16.01.2017 | Power and Electrical Engineering

New technology enables 5-D imaging in live animals, humans

16.01.2017 | Information Technology

Researchers develop environmentally friendly soy air filter

16.01.2017 | Power and Electrical Engineering

VideoLinks
B2B-VideoLinks
More VideoLinks >>>