Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

IBC 2019 – Using Artificial Intelligence for Media Content Analysis

10.09.2019

Whether it is about operating an intelligent media archive, providing real-time subtitles for TV shows or analyzing entire radio or TV programs – with the help of artificial intelligence, media content can be systematically analyzed. This not just facilitates daily routines of media professionals, but also allows offering personalized content to media consumers – and doing so in a privacy-preserving manner. At IBC 2019, taking place September 13 – 17 in Amsterdam, Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS and Fraunhofer Institute for Digital Media Technology IDMT will be presenting various tools for content analysis at Fraunhofer’s booth B80 in hall 8.

Using artificial intelligence (AI) and machine learning algorithms, state-of-the-art technologies can be used to automatically extract metadata from audio, image, video, and text material. Content analysis tools are more and more requested today, as most data inventories are too large for being manually annotated.


With the help of Fraunhofer IAIS’s Audio Mining system, it is possible to crawl through audio or video tracks in order to systematically search for original utterances made by people.

Fraunhofer IAIS


Analyzing not just individual items, but entire programs broadcasted by radio stations is what Fraunhofer IDMT‘s AI-based program analysis tool is made for.

Fraunhofer IDMT

To meet this demand, the data analysis and media technology experts of Fraunhofer IAIS and Fraunhofer IDMT have developed special tools for various application scenarios and media formats. Not just media professionals in editorial departments or archives of TV networks and radio stations, but also consumers can benefit from such tools.

High-capacity Audio Mining system each day analyzes about 2,000 hours of archived content at one of Germany’s major national TV and radio broadcasters

Detecting specific soundbites in recorded audio or video material can be a very tedious endeavor for journalists or editors. With the help of Fraunhofer IAIS’s Audio Mining system, it is possible to crawl through audio or video tracks in order to systematically search for original utterances made by people. The tool takes advantage of deep learning, allowing speech-to-text conversion of material recorded during or for a radio or TV show.

“Each broadcast is completely available as a text file, in which single words or strings of words can be detected within a fraction of a second. For each word, the time it was uttered during the recording is exactly identifiable. Users may then mark a certain word or a string of words in the text in order to get to the respective soundbite and cut it out from the overall recording”, explains Christoph Schmidt, head of the Speech Technologies division at Fraunhofer IAIS.

Another feature offered by the system is speech recognition in combination with speaker clustering, which allows distinguishing utterances of different individuals within a recording. Searching for content within archives thereby becomes substantially easier, as the tool is capable of responding also to complex user requests (such as searching for “statements by Angela Merkel on nuclear power phase-out”).

Likewise, the system allows users to jump to a certain sequence of utterances (made by a certain person during a talk show, for example) simply by a mouse-click. Among the TV and radio networks using the tool on a regular basis is ARD, one of Germany’s major national TV and radio broadcasters featuring also a number of regional branches, where a total of approximately 2,000 hours of archived material needs to be analyzed on a day-to-day basis.

The experts of Fraunhofer IAIS are currently working on developing their Audio Mining technologies further to become a full-blown dialog system – and thereby an intelligent assistant capable of responding to spoken commands or questions.

Besides the analysis of preproduced content, other application scenarios are live events during which speech needs to be converted to text in real time. For example, the live recognition tool is used in the regional parliament of the state of Saxony for providing real-time subtitles during debates. In the future, the tool could be used for different types of live TV broadcasts (such as one-on-one interviews or talk shows), but also by providers of streaming services, relieving media professionals of time-consuming transcription processes.


AI-based program analysis tool for radio broadcaster

Analyzing not just individual items, but entire programs broadcasted by radio stations is what Fraunhofer IDMT‘s AI-based program analysis tool is made for. The tool can e.g. help to trace how a certain news story is aired by different stations (i.e. at what time and with what modifications made to it), to what extent certain program elements recur during a day or week, or to estimate to what extent a program differs from programs of other stations. Radio stations can use this information to optimize their programs accordingly.

One of the features of Fraunhofer IDMT’s tool is partial matching, which detects reuse of jingles, commercial spots, news stories, or pieces of music during a defined period of time. “By looking at the number of repetitions of elements and when they occur during a day, it is possible to make conclusions about a program’s content and to compare programs with each other”, explains Patrick Aichroth, head of the Media Distribution and Security group at Fraunhofer IDMT. For this purpose, the tool also includes music detection and music analysis, in terms of automatically identifying genre, tempo and other attributes.

Privacy-aware personalization services

Radio and TV broadcasters increasingly aim at offering listeners and viewers personalized content, but user privacy also has become an aspect of critical importance. Recommendations are typically generated either on the basis of content metadata (content-based), or on the basis of collaborative user and usage analysis (collaborative-filtering). Each method has its advantages: While the former e.g. allows providing recommendations across media barriers and formats (i.e. images, text, audio and video), the latter supports consideration of individual user feedback.

To leverage the advantages of both methods, the experts of Fraunhofer IDMT combine them to so-called “hybrid recommendation” approaches. In addition, they use a patented method which allows to perform personalization without infringing on the user’s data sovereignty, by strongly decoupling real user identities from pseudonyms which are used for analysis, thereby securely hiding the real identity.

Both Fraunhofer institutes are currently working on bringing their technologies and tools together to multiply the possibilities and application options for the media industry.

Contacts

Fraunhofer Institute for Intelligent Analysis- and Information Systems IAIS
Schloss Birlinghoven
53757 Sankt Augustin, Germany
Silke Loh, Public Relations
pr@iais.fraunhofer.de
Phone +49 2241 14-2829

Fraunhofer Institute for Digital Media Technology IDMT
Ehrenbergstraße 31
98693 Ilmenau, Germany
Julia Hallebach, Public Relations and Marketing
Julia.Hallebach@idmt.fraunhofer.der
Phone +49 3677 467310

Weitere Informationen:

http://www.idmt.fraunhofer.de/m2d Fraunhofer IDMT Technologies and Solutions
http://www.iais.fraunhofer.de/audiomining Fraunhofer IAIS Audio Mining System

Silke Loh | Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS

More articles from Trade Fair News:

nachricht Fraunhofer FHR presents innovative contributions to radar applications in the automotive sector at the IAA
09.09.2019 | Fraunhofer-Institut für Hochfrequenzphysik und Radartechnik FHR

nachricht Next generation video: WDR and Fraunhofer HHI present significantly improved video quality at IFA 2019
03.09.2019 | Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI

All articles from Trade Fair News >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Physicists from Stuttgart prove the existence of a supersolid state of matte

A supersolid is a state of matter that can be described in simplified terms as being solid and liquid at the same time. In recent years, extensive efforts have been devoted to the detection of this exotic quantum matter. A research team led by Tilman Pfau and Tim Langen at the 5th Institute of Physics of the University of Stuttgart has succeeded in proving experimentally that the long-sought supersolid state of matter exists. The researchers report their results in Nature magazine.

In our everyday lives, we are familiar with matter existing in three different states: solid, liquid, or gas. However, if matter is cooled down to extremely...

Im Focus: World record for tandem perovskite-CIGS solar cell

A team headed by Prof. Steve Albrecht from the HZB will present a new world-record tandem solar cell at EU PVSEC, the world's largest international photovoltaic and solar energy conference and exhibition, in Marseille, France on September 11, 2019. This tandem solar cell combines the semiconducting materials perovskite and CIGS and achieves a certified efficiency of 23.26 per cent. One reason for this success lies in the cell’s intermediate layer of organic molecules: they self-organise to cover even rough semiconductor surfaces. Two patents have been filed for these layers.

Perovskite-based solar cells have experienced an incredibly rapid increase in efficiency over the last ten years. The combination of perovskites with classical...

Im Focus: A molecular 'atlas' of animal development

Researchers from the University of Pennsylvania provide a molecular map of every cell in a developing animal embryo

In a paper in Science this week, Penn researchers report the first detailed molecular characterization of how every cell changes during animal embryonic...

Im Focus: Next generation video: WDR and Fraunhofer HHI present significantly improved video quality at IFA 2019

The demand for even higher resolution videos will continue to increase in the coming years. For this reason, the German public service broadcaster WDR and the Fraunhofer Heinrich Hertz Institute HHI will collaborate in the coming months to test the Video Coding possibilities offered by the next international standard VVC/H.266.

VVC/H.266 is the successor standard to HEVC/H.265. The latter is currently the most modern and efficient standard for Video Coding and is used, for example, in...

Im Focus: Nanodiamonds in the brain

The recording of images of the human brain and its therapy in neurodegenerative diseases is still a major challenge in current medical research. The so-called blood-brain barrier, a kind of filter system of the body between the blood system and the central nervous system, constrains the supply of drugs or contrast media that would allow therapy and image acquisition. Scientists at the Max Planck Institute for Polymer Research (MPI-P) have now produced tiny diamonds, so-called "nanodiamonds", which could serve as a platform for both the therapy and diagnosis of brain diseases.

The blood-brain barrier is a physiological boundary layer that works highly selectively and thus protects the brain: On the one hand, pathogens or toxins are...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Interspeech 2019 conference: Alexa and Siri in Graz

04.09.2019 | Event News

AI for Laser Technology Conference: optimizing the use of lasers with artificial intelligence

29.08.2019 | Event News

The power of thought – the key to success: CYBATHLON BCI Series 2019

16.08.2019 | Event News

 
Latest News

Society 5.0: putting humans at the heart of digitalisation

10.09.2019 | Event News

IBC 2019 – Using Artificial Intelligence for Media Content Analysis

10.09.2019 | Trade Fair News

Physicists from Stuttgart prove the existence of a supersolid state of matte

09.09.2019 | Physics and Astronomy

VideoLinks
Science & Research
Overview of more VideoLinks >>>