Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Smartwatches can now track your finger in mid-air using sonar

16.03.2016

As mobile and wearable devices such as smartwatches grow smaller, it gets tougher for people to interact with screens the size of a matchbook.

That could change with a new sonar technology developed by University of Washington computer scientists and electrical engineers that allows you to interact with mobile devices by writing or gesturing on any nearby surface -- a tabletop, a sheet of paper or even in mid-air.


University of Washington computer science and engineering and electrical engineering researchers demonstrate FingerIO, a new technology from CSE's Networks and Mobile Systems Lab that employs sonar to enable users to interact with their smartphones and smartwatches by gesturing or writing on any nearby surface.

Credit: University of Washington Department of Computer Science & Engineering

FingerIO tracks fine-grained finger movements by turning a smartphone or smartwatch into an active sonar system using the device's own microphones and speakers.

Because sound waves travel through fabric and do not require a line of sight, users can even interact with a phone inside a front pocket or a smartwatch hidden under a sweater sleeve.

In a paper to be presented in May at the Association for Computing Machinery's CHI 2016 conference in San Jose, California, the UW team demonstrates that FingerIO can accurately track two-dimensional finger movements to within 8mm, which is sufficiently accurate to interact with today's mobile devices. The work was recognized with an honorable mention award by the conference.

"You can't type very easily onto a smartwatch display, so we wanted to transform a desk or any area around a device into an input surface," said lead author Rajalakshmi Nandakumar, a UW doctoral student in computer science and engineering. "I don't need to instrument my fingers with any other sensors -- I just use my finger to write something on a desk or any other surface and the device can track it with high resolution."

Using FingerIO, one could use the flick of a finger to turn up the volume, press a button, or scroll through menus on a smartphone without touching it, or even write a search command or text in the air rather than typing on a tiny screen.

FingerIO turns a smartwatch or smartphone into a sonar system using the device's own speaker to emit an inaudible sound wave. That signal bounces off the finger, and those "echoes" are recorded by the device's microphones and used to calculate the finger's location in space.

Using sound waves to track finger motion offers several advantages over cameras -- which don't work without line-of-sight when the device is hidden by fabric or another obstructions -- and other technologies like radar that require both custom sensor hardware and greater computing power, said senior author and UW assistant professor of computer science and engineering Shyam Gollakota.

"Acoustic signals are great -- because sound waves travel much slower than the radio waves used in radar, you don't need as much processing bandwidth so everything is simpler," said Gollakota, who directs the UW's Networks and Mobile Systems Lab. "And from a cost perspective, almost every device has a speaker and microphones so you can achieve this without any special hardware."

But sonar echoes are weak and typically not accurate enough to track finger motion at a high resolution. Errors of a few centimeters make it impossible to differentiate between writing individual letters or subtle hand gestures.

The UW researchers employed a type of signal typically used in wireless communication -- called Orthogonal Frequency Division Multiplexing -- and demonstrated that it can be used to achieve high-resolution finger tracking using sound. Their algorithms leverage the properties of OFDM signals to track phase changes in the echoes and correct for any errors in the finger location to achieve sub-centimeter finger tracking.

To test their approach, the researchers created a FingerIO prototype app for Android devices and downloaded it to an off-the-shelf Samsung Galaxy S4 smartphone and a smartwatch customized with two microphones, which are needed to track finger motion in two dimensions. Today's smartwatches typically only have one, which can be used to track a finger in one dimension.

The researchers asked testers to draw shapes such as stars, squiggles or figure 8s on a touchpad next to a smartphone or smartwatch running FingerIO. Then they compared the touchpad tracings to the shapes created by FingerIO's tracking.

The average difference between the drawings and the FingerIO tracings was 0.8 centimeters for the smartphone and 1.2 centimeters for the smartwatch.

"Given that your finger is already a centimeter thick, that's sufficient to accurately interact with the devices," said co-author and electrical engineering graduate student Vikram Iyer.

Next steps for the research team include demonstrating how FingerIO can be used to track multiple fingers moving at the same time, and extending its tracking abilities into three dimensions by adding additional microphones to the devices.

###

The research was funded by the National Science Foundation and Google.

Co-authors include Microsoft Research principal researcher Desney Tan.

FROM: Jennifer Langston
University of Washington
206-543-2580
jlangst@uw.edu

For more information, contact fingerio@cs.washington.edu.

http://www.washington.edu/news/ 

Jennifer Langston | EurekAlert!

Further reports about: computer science finger motion microphones mid-air mobile devices sonar sound waves waves

More articles from Information Technology:

nachricht PhoxTroT: Optical Interconnect Technologies Revolutionized Data Centers and HPC Systems
11.12.2017 | Fraunhofer-Institut für Zuverlässigkeit und Mikrointegration IZM

nachricht Rules for superconductivity mirrored in 'excitonic insulator'
08.12.2017 | Rice University

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Scientists channel graphene to understand filtration and ion transport into cells

Tiny pores at a cell's entryway act as miniature bouncers, letting in some electrically charged atoms--ions--but blocking others. Operating as exquisitely sensitive filters, these "ion channels" play a critical role in biological functions such as muscle contraction and the firing of brain cells.

To rapidly transport the right ions through the cell membrane, the tiny channels rely on a complex interplay between the ions and surrounding molecules,...

Im Focus: Towards data storage at the single molecule level

The miniaturization of the current technology of storage media is hindered by fundamental limits of quantum mechanics. A new approach consists in using so-called spin-crossover molecules as the smallest possible storage unit. Similar to normal hard drives, these special molecules can save information via their magnetic state. A research team from Kiel University has now managed to successfully place a new class of spin-crossover molecules onto a surface and to improve the molecule’s storage capacity. The storage density of conventional hard drives could therefore theoretically be increased by more than one hundred fold. The study has been published in the scientific journal Nano Letters.

Over the past few years, the building blocks of storage media have gotten ever smaller. But further miniaturization of the current technology is hindered by...

Im Focus: Successful Mechanical Testing of Nanowires

With innovative experiments, researchers at the Helmholtz-Zentrums Geesthacht and the Technical University Hamburg unravel why tiny metallic structures are extremely strong

Light-weight and simultaneously strong – porous metallic nanomaterials promise interesting applications as, for instance, for future aeroplanes with enhanced...

Im Focus: Virtual Reality for Bacteria

An interdisciplinary group of researchers interfaced individual bacteria with a computer to build a hybrid bio-digital circuit - Study published in Nature Communications

Scientists at the Institute of Science and Technology Austria (IST Austria) have managed to control the behavior of individual bacteria by connecting them to a...

Im Focus: A space-time sensor for light-matter interactions

Physicists in the Laboratory for Attosecond Physics (run jointly by LMU Munich and the Max Planck Institute for Quantum Optics) have developed an attosecond electron microscope that allows them to visualize the dispersion of light in time and space, and observe the motions of electrons in atoms.

The most basic of all physical interactions in nature is that between light and matter. This interaction takes place in attosecond times (i.e. billionths of a...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

See, understand and experience the work of the future

11.12.2017 | Event News

Innovative strategies to tackle parasitic worms

08.12.2017 | Event News

AKL’18: The opportunities and challenges of digitalization in the laser industry

07.12.2017 | Event News

 
Latest News

Midwife and signpost for photons

11.12.2017 | Physics and Astronomy

How do megacities impact coastal seas? Searching for evidence in Chinese marginal seas

11.12.2017 | Earth Sciences

PhoxTroT: Optical Interconnect Technologies Revolutionized Data Centers and HPC Systems

11.12.2017 | Information Technology

VideoLinks
B2B-VideoLinks
More VideoLinks >>>