Motion recognition method successful for 10 years

Nils Papenberg, research scientist at Fraunhofer Institute for Medical Image Computing MEVIS, is receiving one of the most prestigious awards in the field of machine vision. The 36-year-old is accepting the Koenderink award with three colleagues in Zurich on September 8th.

It is being given at the European Conference on Computer Vision (ECCV), one of the most important conferences in the field. The prize is given every two years and honors publications issued in the last decade that have proven to be of outstanding value for research and industry.

At the University of Saarbrücken with his colleagues Thomas Brox, Andrés Bruhn, and Joachim Weickert, Papenberg developed a mathematical method that made the recognition of movements in movies and picture sequences more precise than ever before.

The method has been revisited by numerous researchers and companies and can be found in applications including contemporary special effects computer programs for the movie industry. Medicine also benefits from this method: the motion recognition helps to fuse medical image data from different devices. Moreover, the award-winning method is being applied and further developed in research projects at Fraunhofer MEVIS.

In 2004, the team presented their new method at the ECCV congress. The question was: how can we make a computer recognize movement on a picture sequence or a movie as accurately as possible? The researchers used street traffic scenes as test sequences: cars stopping at a traffic light and then taking off again – one faster, one slower – some taking turns in another direction.

Older methods observed the cars based on only one parameter – the brightness of the objects in the image, the so-called gray value Papenberg and his colleagues included another parameter: in addition to the gray value their method also considered certain edges of the cars and followed their movement over several consecutive images.

With the help of the additional classification, the algorithm could now recognize the movement with twice as much accuracy as older methods – this applied particularly to the direction of motion of the cars. Professionals were impressed: experts had considered the achievement of such accuracy to be impossible. In addition, scientists were able to describe and confirm a theory about an existing procedure for movement recognition.
Papenberg and his colleagues proved with mathematical accuracy why this approach, called warping, is functional and useful.

In the last decade, their work has been cited nearly 1200 times in the articles of other scientists – a figure that shows the importance of the method for researchers. Meanwhile, it has been further developed by different research groups and implemented in a wide range of applications. In special effects software, the algorithm helps make scenes appear sped-up or in slow motion.

Nils Papenberg transferred in 2004 from Saarbrücken to the University of Lübeck and in 2009 to the Fraunhofer MEVIS Project Group Image Registration. A team of specialists in the fields of mathematics and computer science explores methods for fusing medical images and develops award-winning methods for use in medicine.

The method has been included in software that merges images from two diagnostic procedures (CT and PET). It supports fusing CT and MR images for the implantation of brain pacemakers. As a result, the surgeon is able to recognize and differentiate between bones, pacemaker, and even brain tissue on a monitor. If breast cancer is suspected, tissue samples can be taken in a gentler way by guiding the biopsy needle via ultrasound.

The method helps generate precise localization. The algorithm is also of interest for tumor treatment with particle beams or X-ray: if the tumor is in or on a moving organ, the rays can follow the movements of the organ so that they can reach the cancer as precisely as possible and spare the surrounding healthy tissue.

http://www.mevis.fraunhofer.de/en/news/press-release/article/motion-recognition-…