Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Show your hands: Smartwatches sense hand activity

07.05.2019

Devices that know what your hands are doing could unlock new apps

We've become accustomed to our smartwatches and smartphones sensing what our bodies are doing, be it walking, driving or sleeping. But what about our hands? It turns out that smartwatches, with a few tweaks, can detect a surprising number of things your hands are doing.


Researchers at Carnegie Mellon University's Human-Computer Interaction Institute (HCII) have used a standard smartwatch to figure out when a wearer was typing on a keyboard, washing dishes, petting a dog, playing piano or a number of other hand activities.

Credit: Carnegie Mellon University

Researchers at Carnegie Mellon University's Human-Computer Interaction Institute (HCII) have used a standard smartwatch to figure out when a wearer was typing on a keyboard, washing dishes, petting a dog, pouring from a pitcher or cutting with scissors.

By making a few changes to the watch's operating system, they were able to use its accelerometer to recognize hand motions and, in some cases, bio-acoustic sounds associated with 25 different hand activities at around 95 percent accuracy. And those 25 activities are just the beginning of what might be possible to detect.

"We envision smartwatches as a unique beachhead on the body for capturing rich, everyday activities," said Chris Harrison, assistant professor in the HCII and director of the Future Interfaces Group. "A wide variety of apps could be made smarter and more context-sensitive if our devices knew the activity of our bodies and hands."

Harrison and HCII Ph.D. student Gierad Laput will present their findings on this new sensing capability at CHI 2019, the Association for Computing Machinery's Conference on Human Factors in Computing Systems, May 4-9 in Glasgow, Scotland. A video is available.

Just as smartphones now can block text messages while a user is driving, future devices that sense hand activity might learn not to interrupt someone while they are doing certain work with their hands, such as chopping vegetables or operating power equipment, Laput said. Sensing hand activity also lends itself to health-related apps -- monitoring activities such as brushing teeth, washing hands or smoking a cigarette.

Hand-sensing also might be used by apps that provide feedback to users who are learning a new skill, such as playing a musical instrument, or undergoing physical rehabilitation. Apps might alert users to typing habits that could lead to repetitive strain injury (RSI), or assess the onset of motor impairments such as those associated with Parkinson's disease.

Laput and Harrison began their exploration of hand activity detection by recruiting 50 people to wear specially programmed smartwatches for almost 1,000 hours while going about their daily activities. Periodically, the watches would record hand motion, hand orientation and bio-acoustic information, and then prompt the wearer to describe the hand activity -- shaving, clapping, scratching, putting on lipstick, etc. More than 80 hand activities were labeled in this way, providing a unique dataset.

For now, users must wear the smartwatch on their active arm, rather than the passive (non-dominant) arm where people typically wear wristwatches, for the system to work. Future experiments will explore what events can be detected using the passive arm.

"The 25 hand activities we evaluated are a small fraction of the ways we engage our arms and hands in the real world," Laput said. Future work likely will focus on classes of activities -- those associated with specific activities such as smoking cessation, elder care, or typing and RSI.

###

The Packard Foundation, Sloan Foundation and the Google Ph.D. Fellowship supported this research.

Media Contact

Byron Spice
bspice@cs.cmu.edu
412-268-9068

 @CMUScience

http://www.cmu.edu

Byron Spice | EurekAlert!
Further information:
https://www.cs.cmu.edu/news/show-your-hands-smartwatches-sense-hand-activity

More articles from Information Technology:

nachricht Artificial intelligence is becoming sustainable!
17.02.2020 | Politecnico di Milano

nachricht Skyrmions like it hot: Spin structures are controllable even at high temperatures
14.02.2020 | Johannes Gutenberg-Universität Mainz

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Freiburg researcher investigate the origins of surface texture

Most natural and artificial surfaces are rough: metals and even glasses that appear smooth to the naked eye can look like jagged mountain ranges under the microscope. There is currently no uniform theory about the origin of this roughness despite it being observed on all scales, from the atomic to the tectonic. Scientists suspect that the rough surface is formed by irreversible plastic deformation that occurs in many processes of mechanical machining of components such as milling.

Prof. Dr. Lars Pastewka from the Simulation group at the Department of Microsystems Engineering at the University of Freiburg and his team have simulated such...

Im Focus: Skyrmions like it hot: Spin structures are controllable even at high temperatures

Investigation of the temperature dependence of the skyrmion Hall effect reveals further insights into possible new data storage devices

The joint research project of Johannes Gutenberg University Mainz (JGU) and the Massachusetts Institute of Technology (MIT) that had previously demonstrated...

Im Focus: Making the internet more energy efficient through systemic optimization

Researchers at Chalmers University of Technology, Sweden, recently completed a 5-year research project looking at how to make fibre optic communications systems more energy efficient. Among their proposals are smart, error-correcting data chip circuits, which they refined to be 10 times less energy consumptive. The project has yielded several scientific articles, in publications including Nature Communications.

Streaming films and music, scrolling through social media, and using cloud-based storage services are everyday activities now.

Im Focus: New synthesis methods enhance 3D chemical space for drug discovery

After helping develop a new approach for organic synthesis -- carbon-hydrogen functionalization -- scientists at Emory University are now showing how this approach may apply to drug discovery. Nature Catalysis published their most recent work -- a streamlined process for making a three-dimensional scaffold of keen interest to the pharmaceutical industry.

"Our tools open up whole new chemical space for potential drug targets," says Huw Davies, Emory professor of organic chemistry and senior author of the paper.

Im Focus: Quantum fluctuations sustain the record superconductor

Superconductivity approaching room temperature may be possible in hydrogen-rich compounds at much lower pressures than previously expected

Reaching room-temperature superconductivity is one of the biggest dreams in physics. Its discovery would bring a technological revolution by providing...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

70th Lindau Nobel Laureate Meeting: Around 70 Laureates set to meet with young scientists from approx. 100 countries

12.02.2020 | Event News

11th Advanced Battery Power Conference, March 24-25, 2020 in Münster/Germany

16.01.2020 | Event News

Laser Colloquium Hydrogen LKH2: fast and reliable fuel cell manufacturing

15.01.2020 | Event News

 
Latest News

First research results on the "spectacular meteorite fall" of Flensburg

18.02.2020 | Earth Sciences

Blood and sweat: Wearable medical sensors will get major sensitivity boost

18.02.2020 | Life Sciences

How to mend a broken heart

18.02.2020 | Life Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>