Easier and systematic qualification of AI applications

The “AIQualify” project aims to develop a software framework that supports companies in their efforts to audit AI systems as part of industrial quality control processes.
(c) Rainer Bez / Fraunhofer IPA

A new software framework is to make it easier for companies to approve or audit applications based on Artificial Intelligence (AI). The framework is being jointly developed by Fraunhofer IPA in conjunction with the Institute of Industrial Manufacturing and Management IFF at the University of Stuttgart as part of the “AIQualify” research project of the German Association for Quality (DGQ).

AI-based applications are gaining ground, especially in the fields of industrial image processing and quality control. Such applications can now automate monotonous and time-consuming manual processes very reliably and offer a significant competitive advantage, in particular to small and medium-sized enterprises (SMEs). This is especially the case when the test parts or potential defects are variable and could consequently push conventional rule-based systems to their limits.

However, a lack of transparency and understanding of how these applications actually work leads to uncertainty when it comes to deploying AI. The applications mostly rely on machine learning (ML) processes. The artificial neural networks used here learn independently from large amounts of data. However, even AI experts are rarely able to explain how exactly a result – and in a worst-case scenario, an incorrect result – is produced based on this learning process. It is precisely this situation that leads to businesses erring on the side of caution when it comes to adopting technology of this kind. In addition, legal problems could arise in the event that in future, businesses are forced to comply with additional regulations for the use of AI technologies in the context of the EU AI Act.

Simplifying and supporting AI audits

In order to remedy this and to provide greater security, especially for businesses with little experience in using AI, suitable standards and development methods are required. These would make it easier to verify the suitability or qualification of ML-based AI applications, even without the kind of specialist knowledge that has been necessary up to now.

This is precisely the goal pursued by the “AIQualify” research project with the help of an emerging software framework. A software-based assistance system supports users in defining and formulating test and evaluation criteria. These are centrally bundled in what is known as an assurance case, before being used to subsequently approve the AI application based on these criteria. The basis for this is an audit platform that provides specific audit modules for each development phase of the ML components of the AI application. A modular design is to be used for the platform, with the aim of ensuring that test modules can be easily integrated or expanded. In addition to isolated qualification, the framework can also be integrated iteratively as an element in the development process of an AI system.

Taking the entire development process into consideration

Prof. Marco Huber, who heads up the project, underlines the innovative nature of the resultant approach: “Rather than only taking the finished application into account, we go back much further – right to the very beginning, in fact. Each development phase for an AI application requires decisions that can all influence the result. For this reason, we also make sure to include aspects such as data selection, pre-processing, quality criteria and model selection.”

The software framework therefore enables three types of qualification: 1. by the company itself, 2. by a customer, supplier or partner, and finally 3. through independent institutions. This produces three distinct target groups: First, service providers for ML-based quality control and management, second, manufacturing companies and, third, service providers for conformity testing and auditing. In particular, small and medium-sized enterprises (SMEs) will have the capability of qualifying third-party AI systems. In this way, the aim is for such businesses to be able to evaluate the performance of an AI system even without having their own AI specialists at their disposal.

Evaluating the framework by way of typical applications

Two use cases serve the purpose of testing the software framework in practical terms. The first use case relates to the research context of the project partners, where AI is used for camera-based detection of defective perforated discs. What is so special about this is that, in addition to real camera images, synthetic images of defects can also be created and used. This allows different degrees of severity of the test task to be taken into consideration when it comes to assessing the suitability of the ML components. The second use case comes directly from industrial practice. In addition to supporting the project as a whole, a project committee comprising manufacturing companies, among other organizations, will contribute this use case to the project.

Project overview: AIQualify

Full name: AIQualify – Framework for the qualification of AI systems in industrial quality inspection

Duration: 01 May 2023 to 30 April 2025

Partners: Fraunhofer Institute for Manufacturing Engineering and Automation IPA; Institute of Industrial Manufacturing and Management IFF at the University of Stuttgart

Associated members on the project committee: 36ZERO Vision, Audi, Babtec Informationssysteme, Bosch, EVT, Festool, Maddox AI, preML, scitis.io, sentin and Wickon Hightech. In addition, the “Allianz Industrie 4.0”, a partnership between Bitkom and the German University of Administrative Sciences Speyer, is also supporting the project.

Funding information: The IGF project 22929 BG of the Federation of Quality Research and Science (FQS), was funded by the Federal Ministry for Economic Affairs and Climate Action through the German Federation of Industrial Research Associations (AiF) under the Industrial Collective Research (IGF) programme on the basis of a decision by the German Bundestag.

Wissenschaftliche Ansprechpartner:

Prof. Dr.-Ing. Marco Huber | Tel. +49 711 970-1960 | marco.huber@ipa.fraunhofer.de

Weitere Informationen:



Media Contact

Dr. Karin Röhricht Presse- und Öffentlichkeitsarbeit
Fraunhofer-Institut für Produktionstechnik und Automatisierung IPA

All latest news from the category: Machine Engineering

Machine engineering is one of Germany’s key industries. The importance of this segment has led to the creation of new university degree programs in fields such as production and logistics, process engineering, vehicle/automotive engineering, production engineering and aerospace engineering among others.

innovations-report offers informative reports and articles covering technologies such as automation, motion, power train, energy, conveyor, plastics, lightweight construction, logistics/warehousing, measurement systems, machine tools and control engineering.

Back to home

Comments (0)

Write a comment

Newest articles

Superradiant atoms could push the boundaries of how precisely time can be measured

Superradiant atoms can help us measure time more precisely than ever. In a new study, researchers from the University of Copenhagen present a new method for measuring the time interval,…

Ion thermoelectric conversion devices for near room temperature

The electrode sheet of the thermoelectric device consists of ionic hydrogel, which is sandwiched between the electrodes to form, and the Prussian blue on the electrode undergoes a redox reaction…

Zap Energy achieves 37-million-degree temperatures in a compact device

New publication reports record electron temperatures for a small-scale, sheared-flow-stabilized Z-pinch fusion device. In the nine decades since humans first produced fusion reactions, only a few fusion technologies have demonstrated…

Partners & Sponsors