Professor Otmar Hilliges and his staff at ETH Zurich have developed a new app enabling users to operate their smartphone with gestures. This development expands the range of potential interactions with such devices.
It does seem slightly odd at first: you hold the phone in one hand, and move the other in the air above its built-in camera making gestures that resemble sign language. Sometimes you move your index finger to the left, sometimes to the right. You can spread out your fingers, or imitate a pair of pliers or the firing of a pistol. These gestures are not, however, intended for communicating with deaf people; they are for controlling your smartphone.
By mimicking the firing of a pistol, for example, a user can switch to another browser tab, change the map’s view from satellite to standard, or shoot down enemy planes in a game. Spreading out your fingers magnifies a section of a map or scrolls the page of a book forwards.
All this gesturing wizardry is made possible by a new type of algorithm developed by Jie Song, a Master’s student in the working group headed by by Otmar Hilliges, Professor of Computer Science. The researchers presented the app to an audience of industry professionals at the UIST symposium in Honolulu, Hawaii.
The program uses the smartphone’s built-in camera to register its environment. It does not evaluate depth or colour. The information it does register – the shape of the gesture, the parts of the hand – is reduced to a simple outline that is classified according to stored gestures. The program then executes the command associated with the gesture it observes. The program also recognises the hand’s distance from the camera and warns the user when the hand is either too close or too far away.
“Many movement-recognition programs need plenty of processor and memory power”, explains Hilliges, adding that their new algorithm uses a far smaller portion of computer memory and is thus ideal for smartphones. He believes the application is the first of its kind that can run on a smartphone. The app’s minimal processing footprint means it could also run on smart watches or in augmented-reality glasses.
The program currently recognises six different gestures and executes their corresponding commands. Although the researchers have tested 16 outlines, this is not the app’s theoretical limit. What matters is that gestures generate unambiguous outlines. Gestures that resemble others are not suitable for this application. “To expand its functionality, we’re going to add further classification schemes to the program”, says the ETH researcher.
He is convinced that this new way of operating smartphones greatly increases the range of interactivity. The researcher’s objective is to keep the gestures as simple as possible, so that users can operate their smartphone effortlessly.
But will smartphone users want to adapt to this new style of interaction? Otmar Hilliges is confident they will. Gesture control will not replace touchscreen control, but supplement it. “People got used to operating computer games with their movements.” Touchscreens, Hilliges reminds us, also required a very long adjustment period before making a big impact in consumers’ lives. He is therefore certain that this application – or at least parts of it – will find its way onto the market.
Song J, Sörös G, Pece F, Fanello S, Izadi S, Keskin C, Hilliges O: In-air Gestures Around Unmodified Mobile Devices, ACM User Interface Software and Technology Symposium, Honolulu, Hawaii, 7 October 2014
Otmar Hilliges | Eurek Alert!
New technology enables 5-D imaging in live animals, humans
16.01.2017 | University of Southern California
Fraunhofer FIT announces CloudTeams collaborative software development platform – join it for free
10.01.2017 | Fraunhofer-Institut für Angewandte Informationstechnik FIT
Researchers from the University of Hamburg in Germany, in collaboration with colleagues from the University of Aarhus in Denmark, have synthesized a new superconducting material by growing a few layers of an antiferromagnetic transition-metal chalcogenide on a bismuth-based topological insulator, both being non-superconducting materials.
While superconductivity and magnetism are generally believed to be mutually exclusive, surprisingly, in this new material, superconducting correlations...
Laser-driving of semimetals allows creating novel quasiparticle states within condensed matter systems and switching between different states on ultrafast time scales
Studying properties of fundamental particles in condensed matter systems is a promising approach to quantum field theory. Quasiparticles offer the opportunity...
Among the general public, solar thermal energy is currently associated with dark blue, rectangular collectors on building roofs. Technologies are needed for aesthetically high quality architecture which offer the architect more room for manoeuvre when it comes to low- and plus-energy buildings. With the “ArKol” project, researchers at Fraunhofer ISE together with partners are currently developing two façade collectors for solar thermal energy generation, which permit a high degree of design flexibility: a strip collector for opaque façade sections and a solar thermal blind for transparent sections. The current state of the two developments will be presented at the BAU 2017 trade fair.
As part of the “ArKol – development of architecturally highly integrated façade collectors with heat pipes” project, Fraunhofer ISE together with its partners...
At TU Wien, an alternative for resource intensive formwork for the construction of concrete domes was developed. It is now used in a test dome for the Austrian Federal Railways Infrastructure (ÖBB Infrastruktur).
Concrete shells are efficient structures, but not very resource efficient. The formwork for the construction of concrete domes alone requires a high amount of...
Many pathogens use certain sugar compounds from their host to help conceal themselves against the immune system. Scientists at the University of Bonn have now, in cooperation with researchers at the University of York in the United Kingdom, analyzed the dynamics of a bacterial molecule that is involved in this process. They demonstrate that the protein grabs onto the sugar molecule with a Pac Man-like chewing motion and holds it until it can be used. Their results could help design therapeutics that could make the protein poorer at grabbing and holding and hence compromise the pathogen in the host. The study has now been published in “Biophysical Journal”.
The cells of the mouth, nose and intestinal mucosa produce large quantities of a chemical called sialic acid. Many bacteria possess a special transport system...
10.01.2017 | Event News
09.01.2017 | Event News
05.01.2017 | Event News
17.01.2017 | Earth Sciences
17.01.2017 | Materials Sciences
17.01.2017 | Architecture and Construction