A look inside neural networks
Artificial intelligence (AI) is already firmly embedded in our everyday lives and is conquering more and more territory. For example, voice assistants are already an everyday item in many people’s smartphones, cars and homes. Progress in the field of AI is based primarily on the use of neural networks.
Mimicking the functionality of the human brain, neural networks link mathematically defined units with one another. But in the past it was not known just how a neural network makes decisions. Researchers at the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI and Technische Universität Berlin have developed a technology that reveals the criteria AI systems use when making decisions.
The innovative Spectral Relevance Analysis (SpRAy) method based on Layerwise Relevance Propagation technology provides a first peek inside the “black box”.
Today it’s almost impossible to find an area in which artificial intelligence is irrelevant, whether in manufacturing, advertising or communications. Many companies use learning and networked AI systems, for example to generate precise demand forecasts and to exactly predict customer behavior. This approach can also be used to adjust regional logistics processes. Healthcare also uses specific AI activities, such as prognosis generation on the basis of structured data.
This plays a role for example in image recognition: X-ray images are input into an AI system which then outputs a diagnosis. Proper detection of image content is also crucial to autonomous driving, where traffic signs, trees, pedestrians and cyclists have to be identified with complete accuracy. And this is the crux of the matter:
AI systems have to provide absolutely reliable problem-solving strategies in sensitive application areas such as medicinal diagnostics and in security-critical areas. However, in the past is hasn't been entirely clear how AI systems make decisions. Furthermore, the predictions depend on the quality of the input data.
Researchers at the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI and Technische Universität Berlin have now developed a technology, Layer-wise Relevance Propagation (LRP), which renders the AI forecasts explainable and in doing so reveals unreliable problem solution strategies. A further development of LRP technology, referred to as Spectral Relevance Analysis (SpRAy), identifies and quantifies a broad spectrum of learned decisionmaking behaviors and thus identifies undesirable decisions even in enormous datasets.
In practice the technology identifies the individual input elements which have been used to make a prediction. Thus for example when an image of a tissue sample is input into an AI system, the influence of each individual pixel is quantified in the classification results. In other words, as well as predicting how “malignant” or “benign” the imaged tissue is, the system also provides information on the basis for this classification. “Not only is the result supposed to be correct, the solution strategy is as well.
In the past, AI systems have been treated as black boxes. The systems were trusted to do the right things. With our open-source software, which uses Layer-Wise Relevance Propagation, we’ve succeeded in rendering the solution-finding process of AI systems transparent,” says Dr. Wojciech Samek, head of the "Machine Learning” research group at Fraunhofer HHI. “We’re using LRP to visualize and interpret neural networks and other machine learning models. We use LRP to measure the influence of every input variable in the overall prediction and parse the decisions made by the classifiers,” adds Dr. Klaus-Robert Müller, Professor for Machine Learning at TU Berlin.
Unreliable solution strategies
Trusting the results of neural networks necessarily means understanding how they work. According to the research team’s tests, AI systems don’t always apply the best strategies to reach a solution. For example, one well-known AI system classifies images based on context. It allocated photographs to the category ‘Ship’ when a large amount of water was visible in the picture. It wasn’t solving the actual task of recognizing images of ships, even if in the majority of cases it picked out the right photos. “Many AI algorithms use unreliable strategies and arrive at highly impractical solutions,” says Samek, summarizing the results of the investigations.
Watching neural networks think
The LRP technology decodes the functionality of neural networks and finds out which characteristic features are used, for example to identify a horse as a horse and not as a donkey or a cow. It identifies the information flowing through the system at each node of the network. This makes it possible to investigate even very deep neural networks.
The Fraunhofer HHI and TU Berlin research teams are currently formulating new algorithms for the investigation of further questions in order to make AI systems even more reliable and robust. The project partners have published their research results in the journal Nature Communications (see link below).
Kathleen Schröter | Fraunhofer Research News
NIST-led team develops tiny low-energy device to rapidly reroute light in computer chips
15.11.2019 | National Institute of Standards and Technology (NIST)
Fraunhofer Radio Technology becomes part of the worldwide Telecom Infra Project (TIP)
14.11.2019 | Fraunhofer-Institut für Angewandte Informationstechnik FIT
Conventional light microscopes cannot distinguish structures when they are separated by a distance smaller than, roughly, the wavelength of light. Superresolution microscopy, developed since the 1980s, lifts this limitation, using fluorescent moieties. Scientists at the Max Planck Institute for Polymer Research have now discovered that graphene nano-molecules can be used to improve this microscopy technique. These graphene nano-molecules offer a number of substantial advantages over the materials previously used, making superresolution microscopy even more versatile.
Microscopy is an important investigation method, in physics, biology, medicine, and many other sciences. However, it has one disadvantage: its resolution is...
Nanooptical traps are a promising building block for quantum technologies. Austrian and German scientists have now removed an important obstacle to their practical use. They were able to show that a special form of mechanical vibration heats trapped particles in a very short time and knocks them out of the trap.
By controlling individual atoms, quantum properties can be investigated and made usable for technological applications. For about ten years, physicists have...
An international team of scientists, including three researchers from New Jersey Institute of Technology (NJIT), has shed new light on one of the central mysteries of solar physics: how energy from the Sun is transferred to the star's upper atmosphere, heating it to 1 million degrees Fahrenheit and higher in some regions, temperatures that are vastly hotter than the Sun's surface.
With new images from NJIT's Big Bear Solar Observatory (BBSO), the researchers have revealed in groundbreaking, granular detail what appears to be a likely...
The Fraunhofer Institute for Manufacturing Technology and Advanced Materials IFAM in Dresden has succeeded in using Selective Electron Beam Melting (SEBM) to...
Carbon nanotubes (CNTs) are valuable for a wide variety of applications. Made of graphene sheets rolled into tubes 10,000 times smaller than a human hair, CNTs have an exceptional strength-to-mass ratio and excellent thermal and electrical properties. These features make them ideal for a range of applications, including supercapacitors, interconnects, adhesives, particle trapping and structural color.
New research reveals even more potential for CNTs: as a coating, they can both repel and hold water in place, a useful property for applications like printing,...
15.11.2019 | Event News
15.11.2019 | Event News
05.11.2019 | Event News
20.11.2019 | Life Sciences
20.11.2019 | Physics and Astronomy
20.11.2019 | Health and Medicine