Devices that know what your hands are doing could unlock new apps
We've become accustomed to our smartwatches and smartphones sensing what our bodies are doing, be it walking, driving or sleeping. But what about our hands? It turns out that smartwatches, with a few tweaks, can detect a surprising number of things your hands are doing.
Researchers at Carnegie Mellon University's Human-Computer Interaction Institute (HCII) have used a standard smartwatch to figure out when a wearer was typing on a keyboard, washing dishes, petting a dog, pouring from a pitcher or cutting with scissors.
By making a few changes to the watch's operating system, they were able to use its accelerometer to recognize hand motions and, in some cases, bio-acoustic sounds associated with 25 different hand activities at around 95 percent accuracy. And those 25 activities are just the beginning of what might be possible to detect.
"We envision smartwatches as a unique beachhead on the body for capturing rich, everyday activities," said Chris Harrison, assistant professor in the HCII and director of the Future Interfaces Group. "A wide variety of apps could be made smarter and more context-sensitive if our devices knew the activity of our bodies and hands."
Harrison and HCII Ph.D. student Gierad Laput will present their findings on this new sensing capability at CHI 2019, the Association for Computing Machinery's Conference on Human Factors in Computing Systems, May 4-9 in Glasgow, Scotland. A video is available.
Just as smartphones now can block text messages while a user is driving, future devices that sense hand activity might learn not to interrupt someone while they are doing certain work with their hands, such as chopping vegetables or operating power equipment, Laput said. Sensing hand activity also lends itself to health-related apps -- monitoring activities such as brushing teeth, washing hands or smoking a cigarette.
Hand-sensing also might be used by apps that provide feedback to users who are learning a new skill, such as playing a musical instrument, or undergoing physical rehabilitation. Apps might alert users to typing habits that could lead to repetitive strain injury (RSI), or assess the onset of motor impairments such as those associated with Parkinson's disease.
Laput and Harrison began their exploration of hand activity detection by recruiting 50 people to wear specially programmed smartwatches for almost 1,000 hours while going about their daily activities. Periodically, the watches would record hand motion, hand orientation and bio-acoustic information, and then prompt the wearer to describe the hand activity -- shaving, clapping, scratching, putting on lipstick, etc. More than 80 hand activities were labeled in this way, providing a unique dataset.
For now, users must wear the smartwatch on their active arm, rather than the passive (non-dominant) arm where people typically wear wristwatches, for the system to work. Future experiments will explore what events can be detected using the passive arm.
"The 25 hand activities we evaluated are a small fraction of the ways we engage our arms and hands in the real world," Laput said. Future work likely will focus on classes of activities -- those associated with specific activities such as smoking cessation, elder care, or typing and RSI.
The Packard Foundation, Sloan Foundation and the Google Ph.D. Fellowship supported this research.
Byron Spice | EurekAlert!
Researchers build transistor-like gate for quantum information processing -- with qudits
17.07.2019 | Purdue University
New DFG Research Group "Metrology for THz Communications"
17.07.2019 | Technische Universität Braunschweig
Adjusting the thermal conductivity of materials is one of the challenges nanoscience is currently facing. Together with colleagues from the Netherlands and Spain, researchers from the University of Basel have shown that the atomic vibrations that determine heat generation in nanowires can be controlled through the arrangement of atoms alone. The scientists will publish the results shortly in the journal Nano Letters.
In the electronics and computer industry, components are becoming ever smaller and more powerful. However, there are problems with the heat generation. It is...
Scientists have visualised the electronic structure in a microelectronic device for the first time, opening up opportunities for finely-tuned high performance electronic devices.
Physicists from the University of Warwick and the University of Washington have developed a technique to measure the energy and momentum of electrons in...
Scientists at the University Würzburg and University Hospital of Würzburg found that megakaryocytes act as “bouncers” and thus modulate bone marrow niche properties and cell migration dynamics. The study was published in July in the Journal “Haematologica”.
Hematopoiesis is the process of forming blood cells, which occurs predominantly in the bone marrow. The bone marrow produces all types of blood cells: red...
For some phenomena in quantum many-body physics several competing theories exist. But which of them describes a quantum phenomenon best? A team of researchers from the Technical University of Munich (TUM) and Harvard University in the United States has now successfully deployed artificial neural networks for image analysis of quantum systems.
Is that a dog or a cat? Such a classification is a prime example of machine learning: artificial neural networks can be trained to analyze images by looking...
An international research group led by scientists from the University of Bayreuth has produced a previously unknown material: Rhenium nitride pernitride. Thanks to combining properties that were previously considered incompatible, it looks set to become highly attractive for technological applications. Indeed, it is a super-hard metallic conductor that can withstand extremely high pressures like a diamond. A process now developed in Bayreuth opens up the possibility of producing rhenium nitride pernitride and other technologically interesting materials in sufficiently large quantity for their properties characterisation. The new findings are presented in "Nature Communications".
The possibility of finding a compound that was metallically conductive, super-hard, and ultra-incompressible was long considered unlikely in science. It was...
24.06.2019 | Event News
29.04.2019 | Event News
17.04.2019 | Event News
19.07.2019 | Physics and Astronomy
19.07.2019 | Physics and Astronomy
19.07.2019 | Earth Sciences