Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:


IBC 2019 – Using Artificial Intelligence for Media Content Analysis


Whether it is about operating an intelligent media archive, providing real-time subtitles for TV shows or analyzing entire radio or TV programs – with the help of artificial intelligence, media content can be systematically analyzed. This not just facilitates daily routines of media professionals, but also allows offering personalized content to media consumers – and doing so in a privacy-preserving manner. At IBC 2019, taking place September 13 – 17 in Amsterdam, Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS and Fraunhofer Institute for Digital Media Technology IDMT will be presenting various tools for content analysis at Fraunhofer’s booth B80 in hall 8.

Using artificial intelligence (AI) and machine learning algorithms, state-of-the-art technologies can be used to automatically extract metadata from audio, image, video, and text material. Content analysis tools are more and more requested today, as most data inventories are too large for being manually annotated.

With the help of Fraunhofer IAIS’s Audio Mining system, it is possible to crawl through audio or video tracks in order to systematically search for original utterances made by people.

Fraunhofer IAIS

Analyzing not just individual items, but entire programs broadcasted by radio stations is what Fraunhofer IDMT‘s AI-based program analysis tool is made for.

Fraunhofer IDMT

To meet this demand, the data analysis and media technology experts of Fraunhofer IAIS and Fraunhofer IDMT have developed special tools for various application scenarios and media formats. Not just media professionals in editorial departments or archives of TV networks and radio stations, but also consumers can benefit from such tools.

High-capacity Audio Mining system each day analyzes about 2,000 hours of archived content at one of Germany’s major national TV and radio broadcasters

Detecting specific soundbites in recorded audio or video material can be a very tedious endeavor for journalists or editors. With the help of Fraunhofer IAIS’s Audio Mining system, it is possible to crawl through audio or video tracks in order to systematically search for original utterances made by people. The tool takes advantage of deep learning, allowing speech-to-text conversion of material recorded during or for a radio or TV show.

“Each broadcast is completely available as a text file, in which single words or strings of words can be detected within a fraction of a second. For each word, the time it was uttered during the recording is exactly identifiable. Users may then mark a certain word or a string of words in the text in order to get to the respective soundbite and cut it out from the overall recording”, explains Christoph Schmidt, head of the Speech Technologies division at Fraunhofer IAIS.

Another feature offered by the system is speech recognition in combination with speaker clustering, which allows distinguishing utterances of different individuals within a recording. Searching for content within archives thereby becomes substantially easier, as the tool is capable of responding also to complex user requests (such as searching for “statements by Angela Merkel on nuclear power phase-out”).

Likewise, the system allows users to jump to a certain sequence of utterances (made by a certain person during a talk show, for example) simply by a mouse-click. Among the TV and radio networks using the tool on a regular basis is ARD, one of Germany’s major national TV and radio broadcasters featuring also a number of regional branches, where a total of approximately 2,000 hours of archived material needs to be analyzed on a day-to-day basis.

The experts of Fraunhofer IAIS are currently working on developing their Audio Mining technologies further to become a full-blown dialog system – and thereby an intelligent assistant capable of responding to spoken commands or questions.

Besides the analysis of preproduced content, other application scenarios are live events during which speech needs to be converted to text in real time. For example, the live recognition tool is used in the regional parliament of the state of Saxony for providing real-time subtitles during debates. In the future, the tool could be used for different types of live TV broadcasts (such as one-on-one interviews or talk shows), but also by providers of streaming services, relieving media professionals of time-consuming transcription processes.

AI-based program analysis tool for radio broadcaster

Analyzing not just individual items, but entire programs broadcasted by radio stations is what Fraunhofer IDMT‘s AI-based program analysis tool is made for. The tool can e.g. help to trace how a certain news story is aired by different stations (i.e. at what time and with what modifications made to it), to what extent certain program elements recur during a day or week, or to estimate to what extent a program differs from programs of other stations. Radio stations can use this information to optimize their programs accordingly.

One of the features of Fraunhofer IDMT’s tool is partial matching, which detects reuse of jingles, commercial spots, news stories, or pieces of music during a defined period of time. “By looking at the number of repetitions of elements and when they occur during a day, it is possible to make conclusions about a program’s content and to compare programs with each other”, explains Patrick Aichroth, head of the Media Distribution and Security group at Fraunhofer IDMT. For this purpose, the tool also includes music detection and music analysis, in terms of automatically identifying genre, tempo and other attributes.

Privacy-aware personalization services

Radio and TV broadcasters increasingly aim at offering listeners and viewers personalized content, but user privacy also has become an aspect of critical importance. Recommendations are typically generated either on the basis of content metadata (content-based), or on the basis of collaborative user and usage analysis (collaborative-filtering). Each method has its advantages: While the former e.g. allows providing recommendations across media barriers and formats (i.e. images, text, audio and video), the latter supports consideration of individual user feedback.

To leverage the advantages of both methods, the experts of Fraunhofer IDMT combine them to so-called “hybrid recommendation” approaches. In addition, they use a patented method which allows to perform personalization without infringing on the user’s data sovereignty, by strongly decoupling real user identities from pseudonyms which are used for analysis, thereby securely hiding the real identity.

Both Fraunhofer institutes are currently working on bringing their technologies and tools together to multiply the possibilities and application options for the media industry.


Fraunhofer Institute for Intelligent Analysis- and Information Systems IAIS
Schloss Birlinghoven
53757 Sankt Augustin, Germany
Silke Loh, Public Relations
Phone +49 2241 14-2829

Fraunhofer Institute for Digital Media Technology IDMT
Ehrenbergstraße 31
98693 Ilmenau, Germany
Julia Hallebach, Public Relations and Marketing
Phone +49 3677 467310

Weitere Informationen: Fraunhofer IDMT Technologies and Solutions Fraunhofer IAIS Audio Mining System

Silke Loh | Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS

All articles from Trade Fair News >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: TU Graz Researchers synthesize nanoparticles tailored for special applications

“Core-shell” clusters pave the way for new efficient nanomaterials that make catalysts, magnetic and laser sensors or measuring devices for detecting electromagnetic radiation more efficient.

Whether in innovative high-tech materials, more powerful computer chips, pharmaceuticals or in the field of renewable energies, nanoparticles – smallest...

Im Focus: Tailored light inspired by nature

An international research team with Prof. Cornelia Denz from the Institute of Applied Physics at the University of Münster develop for the first time light fields using caustics that do not change during propagation. With the new method, the physicists cleverly exploit light structures that can be seen in rainbows or when light is transmitted through drinking glasses.

Modern applications as high resolution microsopy or micro- or nanoscale material processing require customized laser beams that do not change during...

Im Focus: NYUAD astrophysicist investigates the possibility of life below the surface of Mars

  • A rover expected to explore below the surface of Mars in 2022 has the potential to provide more insights
  • The findings published in Scientific Reports, Springer Nature suggests the presence of traces of water on Mars, raising the question of the possibility of a life-supporting environment

Although no life has been detected on the Martian surface, a new study from astrophysicist and research scientist at the Center for Space Science at NYU Abu...

Im Focus: Manipulating non-magnetic atoms in a chromium halide enables tuning of magnetic properties

New approach creates synthetic layered magnets with unprecedented level of control over their magnetic properties

The magnetic properties of a chromium halide can be tuned by manipulating the non-magnetic atoms in the material, a team, led by Boston College researchers,...

Im Focus: A new method to significantly increase the range and stability of optical tweezers

Scientists of Tomsk Polytechnic University jointly with a team of the V.E. Zuev Institute of Atmospheric Optics of the Siberian Branch of the Russian Academy of Sciences have discovered a method to increase the operation range of optical traps also known

Optical tweezers are a device which uses a laser beam to move micron-sized objects such as living cells, proteins, and molecules. In 2018, the American...

All Focus news of the innovation-report >>>



Industry & Economy
Event News

“Conference on Laser Polishing – LaP 2020”: The final touches for surfaces

23.07.2020 | Event News

Conference radar for cybersecurity

21.07.2020 | Event News

Contact Tracing Apps against COVID-19: German National Academy Leopoldina hosts international virtual panel discussion

07.07.2020 | Event News

Latest News

Share and browse technologies, research and best practices on COVID-19

03.08.2020 | Information Technology

Strong evidence – Essential regulatory gene for the formation of heart valves discovered

03.08.2020 | Life Sciences

Understanding collective behavior in networks better

03.08.2020 | Life Sciences

Science & Research
Overview of more VideoLinks >>>