Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:


IBC 2019 – Using Artificial Intelligence for Media Content Analysis


Whether it is about operating an intelligent media archive, providing real-time subtitles for TV shows or analyzing entire radio or TV programs – with the help of artificial intelligence, media content can be systematically analyzed. This not just facilitates daily routines of media professionals, but also allows offering personalized content to media consumers – and doing so in a privacy-preserving manner. At IBC 2019, taking place September 13 – 17 in Amsterdam, Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS and Fraunhofer Institute for Digital Media Technology IDMT will be presenting various tools for content analysis at Fraunhofer’s booth B80 in hall 8.

Using artificial intelligence (AI) and machine learning algorithms, state-of-the-art technologies can be used to automatically extract metadata from audio, image, video, and text material. Content analysis tools are more and more requested today, as most data inventories are too large for being manually annotated.

With the help of Fraunhofer IAIS’s Audio Mining system, it is possible to crawl through audio or video tracks in order to systematically search for original utterances made by people.

Fraunhofer IAIS

Analyzing not just individual items, but entire programs broadcasted by radio stations is what Fraunhofer IDMT‘s AI-based program analysis tool is made for.

Fraunhofer IDMT

To meet this demand, the data analysis and media technology experts of Fraunhofer IAIS and Fraunhofer IDMT have developed special tools for various application scenarios and media formats. Not just media professionals in editorial departments or archives of TV networks and radio stations, but also consumers can benefit from such tools.

High-capacity Audio Mining system each day analyzes about 2,000 hours of archived content at one of Germany’s major national TV and radio broadcasters

Detecting specific soundbites in recorded audio or video material can be a very tedious endeavor for journalists or editors. With the help of Fraunhofer IAIS’s Audio Mining system, it is possible to crawl through audio or video tracks in order to systematically search for original utterances made by people. The tool takes advantage of deep learning, allowing speech-to-text conversion of material recorded during or for a radio or TV show.

“Each broadcast is completely available as a text file, in which single words or strings of words can be detected within a fraction of a second. For each word, the time it was uttered during the recording is exactly identifiable. Users may then mark a certain word or a string of words in the text in order to get to the respective soundbite and cut it out from the overall recording”, explains Christoph Schmidt, head of the Speech Technologies division at Fraunhofer IAIS.

Another feature offered by the system is speech recognition in combination with speaker clustering, which allows distinguishing utterances of different individuals within a recording. Searching for content within archives thereby becomes substantially easier, as the tool is capable of responding also to complex user requests (such as searching for “statements by Angela Merkel on nuclear power phase-out”).

Likewise, the system allows users to jump to a certain sequence of utterances (made by a certain person during a talk show, for example) simply by a mouse-click. Among the TV and radio networks using the tool on a regular basis is ARD, one of Germany’s major national TV and radio broadcasters featuring also a number of regional branches, where a total of approximately 2,000 hours of archived material needs to be analyzed on a day-to-day basis.

The experts of Fraunhofer IAIS are currently working on developing their Audio Mining technologies further to become a full-blown dialog system – and thereby an intelligent assistant capable of responding to spoken commands or questions.

Besides the analysis of preproduced content, other application scenarios are live events during which speech needs to be converted to text in real time. For example, the live recognition tool is used in the regional parliament of the state of Saxony for providing real-time subtitles during debates. In the future, the tool could be used for different types of live TV broadcasts (such as one-on-one interviews or talk shows), but also by providers of streaming services, relieving media professionals of time-consuming transcription processes.

AI-based program analysis tool for radio broadcaster

Analyzing not just individual items, but entire programs broadcasted by radio stations is what Fraunhofer IDMT‘s AI-based program analysis tool is made for. The tool can e.g. help to trace how a certain news story is aired by different stations (i.e. at what time and with what modifications made to it), to what extent certain program elements recur during a day or week, or to estimate to what extent a program differs from programs of other stations. Radio stations can use this information to optimize their programs accordingly.

One of the features of Fraunhofer IDMT’s tool is partial matching, which detects reuse of jingles, commercial spots, news stories, or pieces of music during a defined period of time. “By looking at the number of repetitions of elements and when they occur during a day, it is possible to make conclusions about a program’s content and to compare programs with each other”, explains Patrick Aichroth, head of the Media Distribution and Security group at Fraunhofer IDMT. For this purpose, the tool also includes music detection and music analysis, in terms of automatically identifying genre, tempo and other attributes.

Privacy-aware personalization services

Radio and TV broadcasters increasingly aim at offering listeners and viewers personalized content, but user privacy also has become an aspect of critical importance. Recommendations are typically generated either on the basis of content metadata (content-based), or on the basis of collaborative user and usage analysis (collaborative-filtering). Each method has its advantages: While the former e.g. allows providing recommendations across media barriers and formats (i.e. images, text, audio and video), the latter supports consideration of individual user feedback.

To leverage the advantages of both methods, the experts of Fraunhofer IDMT combine them to so-called “hybrid recommendation” approaches. In addition, they use a patented method which allows to perform personalization without infringing on the user’s data sovereignty, by strongly decoupling real user identities from pseudonyms which are used for analysis, thereby securely hiding the real identity.

Both Fraunhofer institutes are currently working on bringing their technologies and tools together to multiply the possibilities and application options for the media industry.


Fraunhofer Institute for Intelligent Analysis- and Information Systems IAIS
Schloss Birlinghoven
53757 Sankt Augustin, Germany
Silke Loh, Public Relations
Phone +49 2241 14-2829

Fraunhofer Institute for Digital Media Technology IDMT
Ehrenbergstraße 31
98693 Ilmenau, Germany
Julia Hallebach, Public Relations and Marketing
Phone +49 3677 467310

Weitere Informationen: Fraunhofer IDMT Technologies and Solutions Fraunhofer IAIS Audio Mining System

Silke Loh | Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS

More articles from Trade Fair News:

nachricht DYNAFLEX® at e-World 2020
23.01.2020 | Fraunhofer-Institut für Umwelt-, Sicherheits- und Energietechnik UMSICHT

nachricht Medica 2019: Arteriosclerosis - new technologies help to find proper catheters and location of vasoconstriction
11.11.2019 | Technische Universität Kaiserslautern

All articles from Trade Fair News >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Integrate Micro Chips for electronic Skin

Researchers from Dresden and Osaka present the first fully integrated flexible electronics made of magnetic sensors and organic circuits which opens the path towards the development of electronic skin.

Human skin is a fascinating and multifunctional organ with unique properties originating from its flexible and compliant nature. It allows for interfacing with...

Im Focus: Dresden researchers discover resistance mechanism in aggressive cancer

Protease blocks guardian function against uncontrolled cell division

Researchers of the Carl Gustav Carus University Hospital Dresden at the National Center for Tumor Diseases Dresden (NCT/UCC), together with an international...

Im Focus: New roles found for Huntington's disease protein

Crucial role in synapse formation could be new avenue toward treatment

A Duke University research team has identified a new function of a gene called huntingtin, a mutation of which underlies the progressive neurodegenerative...

Im Focus: A new look at 'strange metals'

For years, a new synthesis method has been developed at TU Wien (Vienna) to unlock the secrets of "strange metals". Now a breakthrough has been achieved. The results have been published in "Science".

Superconductors allow electrical current to flow without any resistance - but only below a certain critical temperature. Many materials have to be cooled down...

Im Focus: Programmable nests for cells

KIT researchers develop novel composites of DNA, silica particles, and carbon nanotubes -- Properties can be tailored to various applications

Using DNA, smallest silica particles, and carbon nanotubes, researchers of Karlsruhe Institute of Technology (KIT) developed novel programmable materials....

All Focus news of the innovation-report >>>



Industry & Economy
Event News

11th Advanced Battery Power Conference, March 24-25, 2020 in Münster/Germany

16.01.2020 | Event News

Laser Colloquium Hydrogen LKH2: fast and reliable fuel cell manufacturing

15.01.2020 | Event News

„Advanced Battery Power“- Conference, Contributions are welcome!

07.01.2020 | Event News

Latest News

Researchers discover vaccine to strengthen the immune system of plants

24.01.2020 | Life Sciences

Brain-cell helpers powered by norepinephrine during fear-memory formation

24.01.2020 | Life Sciences

Engineered capillaries model traffic in tiny blood vessels

24.01.2020 | Life Sciences

Science & Research
Overview of more VideoLinks >>>