Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Smartphone understands gestures

09.10.2014

Professor Otmar Hilliges and his staff at ETH Zurich have developed a new app enabling users to operate their smartphone with gestures. This development expands the range of potential interactions with such devices.

It does seem slightly odd at first: you hold the phone in one hand, and move the other in the air above its built-in camera making gestures that resemble sign language. Sometimes you move your index finger to the left, sometimes to the right. You can spread out your fingers, or imitate a pair of pliers or the firing of a pistol. These gestures are not, however, intended for communicating with deaf people; they are for controlling your smartphone.


Reminiscent of sign language: gesture control significantly expands the range of smartphone functionality. (Screenshot: ETH Zurich)

By mimicking the firing of a pistol, for example, a user can switch to another browser tab, change the map’s view from satellite to standard, or shoot down enemy planes in a game. Spreading out your fingers magnifies a section of a map or scrolls the page of a book forwards.

All this gesturing wizardry is made possible by a new type of algorithm developed by Jie Song, a Master’s student in the working group headed by by Otmar Hilliges, Professor of Computer Science. The researchers presented the app to an audience of industry professionals at the UIST symposium in Honolulu, Hawaii.

Intelligent programming uses computer memory

The program uses the smartphone’s built-in camera to register its environment. It does not evaluate depth or colour. The information it does register – the shape of the gesture, the parts of the hand – is reduced to a simple outline that is classified according to stored gestures. The program then executes the command associated with the gesture it observes. The program also recognises the hand’s distance from the camera and warns the user when the hand is either too close or too far away.

“Many movement-recognition programs need plenty of processor and memory power”, explains Hilliges, adding that their new algorithm uses a far smaller portion of computer memory and is thus ideal for smartphones. He believes the application is the first of its kind that can run on a smartphone. The app’s minimal processing footprint means it could also run on smart watches or in augmented-reality glasses.

More control

The program currently recognises six different gestures and executes their corresponding commands. Although the researchers have tested 16 outlines, this is not the app’s theoretical limit. What matters is that gestures generate unambiguous outlines. Gestures that resemble others are not suitable for this application. “To expand its functionality, we’re going to add further classification schemes to the program”, says the ETH researcher.

He is convinced that this new way of operating smartphones greatly increases the range of interactivity. The researcher’s objective is to keep the gestures as simple as possible, so that users can operate their smartphone effortlessly.

But will smartphone users want to adapt to this new style of interaction? Otmar Hilliges is confident they will. Gesture control will not replace touchscreen control, but supplement it. “People got used to operating computer games with their movements.” Touchscreens, Hilliges reminds us, also required a very long adjustment period before making a big impact in consumers’ lives. He is therefore certain that this application – or at least parts of it – will find its way onto the market.

Reference

Song J, Sörös G, Pece F, Fanello S, Izadi S, Keskin C, Hilliges O: In-air Gestures Around Unmodified Mobile Devices, ACM User Interface Software and Technology Symposium, Honolulu, Hawaii, 7 October 2014

Otmar Hilliges | Eurek Alert!
Further information:
https://www.ethz.ch/en/news-and-events/eth-news/news/2014/10/smartphone-understands-gestures.html

Further reports about: ETH Zurich Interface Smartphone algorithm computer memory gesture gestures interactivity movements

More articles from Information Technology:

nachricht Cloud technology: Dynamic certificates make cloud service providers more secure
15.01.2018 | Technische Universität München

nachricht New discovery could improve brain-like memory and computing
10.01.2018 | University of Minnesota

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Artificial agent designs quantum experiments

On the way to an intelligent laboratory, physicists from Innsbruck and Vienna present an artificial agent that autonomously designs quantum experiments. In initial experiments, the system has independently (re)discovered experimental techniques that are nowadays standard in modern quantum optical laboratories. This shows how machines could play a more creative role in research in the future.

We carry smartphones in our pockets, the streets are dotted with semi-autonomous cars, but in the research laboratory experiments are still being designed by...

Im Focus: Scientists decipher key principle behind reaction of metalloenzymes

So-called pre-distorted states accelerate photochemical reactions too

What enables electrons to be transferred swiftly, for example during photosynthesis? An interdisciplinary team of researchers has worked out the details of how...

Im Focus: The first precise measurement of a single molecule's effective charge

For the first time, scientists have precisely measured the effective electrical charge of a single molecule in solution. This fundamental insight of an SNSF Professor could also pave the way for future medical diagnostics.

Electrical charge is one of the key properties that allows molecules to interact. Life itself depends on this phenomenon: many biological processes involve...

Im Focus: Paradigm shift in Paris: Encouraging an holistic view of laser machining

At the JEC World Composite Show in Paris in March 2018, the Fraunhofer Institute for Laser Technology ILT will be focusing on the latest trends and innovations in laser machining of composites. Among other things, researchers at the booth shared with the Aachen Center for Integrative Lightweight Production (AZL) will demonstrate how lasers can be used for joining, structuring, cutting and drilling composite materials.

No other industry has attracted as much public attention to composite materials as the automotive industry, which along with the aerospace industry is a driver...

Im Focus: Room-temperature multiferroic thin films and their properties

Scientists at Tokyo Institute of Technology (Tokyo Tech) and Tohoku University have developed high-quality GFO epitaxial films and systematically investigated their ferroelectric and ferromagnetic properties. They also demonstrated the room-temperature magnetocapacitance effects of these GFO thin films.

Multiferroic materials show magnetically driven ferroelectricity. They are attracting increasing attention because of their fascinating properties such as...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

10th International Symposium: “Advanced Battery Power – Kraftwerk Batterie” Münster, 10-11 April 2018

08.01.2018 | Event News

See, understand and experience the work of the future

11.12.2017 | Event News

Innovative strategies to tackle parasitic worms

08.12.2017 | Event News

 
Latest News

Let the good tubes roll

19.01.2018 | Materials Sciences

How cancer metastasis happens: Researchers reveal a key mechanism

19.01.2018 | Health and Medicine

Meteoritic stardust unlocks timing of supernova dust formation

19.01.2018 | Physics and Astronomy

VideoLinks
B2B-VideoLinks
More VideoLinks >>>