Devices that know what your hands are doing could unlock new apps
We've become accustomed to our smartwatches and smartphones sensing what our bodies are doing, be it walking, driving or sleeping. But what about our hands? It turns out that smartwatches, with a few tweaks, can detect a surprising number of things your hands are doing.
Researchers at Carnegie Mellon University's Human-Computer Interaction Institute (HCII) have used a standard smartwatch to figure out when a wearer was typing on a keyboard, washing dishes, petting a dog, pouring from a pitcher or cutting with scissors.
By making a few changes to the watch's operating system, they were able to use its accelerometer to recognize hand motions and, in some cases, bio-acoustic sounds associated with 25 different hand activities at around 95 percent accuracy. And those 25 activities are just the beginning of what might be possible to detect.
"We envision smartwatches as a unique beachhead on the body for capturing rich, everyday activities," said Chris Harrison, assistant professor in the HCII and director of the Future Interfaces Group. "A wide variety of apps could be made smarter and more context-sensitive if our devices knew the activity of our bodies and hands."
Harrison and HCII Ph.D. student Gierad Laput will present their findings on this new sensing capability at CHI 2019, the Association for Computing Machinery's Conference on Human Factors in Computing Systems, May 4-9 in Glasgow, Scotland. A video is available.
Just as smartphones now can block text messages while a user is driving, future devices that sense hand activity might learn not to interrupt someone while they are doing certain work with their hands, such as chopping vegetables or operating power equipment, Laput said. Sensing hand activity also lends itself to health-related apps -- monitoring activities such as brushing teeth, washing hands or smoking a cigarette.
Hand-sensing also might be used by apps that provide feedback to users who are learning a new skill, such as playing a musical instrument, or undergoing physical rehabilitation. Apps might alert users to typing habits that could lead to repetitive strain injury (RSI), or assess the onset of motor impairments such as those associated with Parkinson's disease.
Laput and Harrison began their exploration of hand activity detection by recruiting 50 people to wear specially programmed smartwatches for almost 1,000 hours while going about their daily activities. Periodically, the watches would record hand motion, hand orientation and bio-acoustic information, and then prompt the wearer to describe the hand activity -- shaving, clapping, scratching, putting on lipstick, etc. More than 80 hand activities were labeled in this way, providing a unique dataset.
For now, users must wear the smartwatch on their active arm, rather than the passive (non-dominant) arm where people typically wear wristwatches, for the system to work. Future experiments will explore what events can be detected using the passive arm.
"The 25 hand activities we evaluated are a small fraction of the ways we engage our arms and hands in the real world," Laput said. Future work likely will focus on classes of activities -- those associated with specific activities such as smoking cessation, elder care, or typing and RSI.
The Packard Foundation, Sloan Foundation and the Google Ph.D. Fellowship supported this research.
Byron Spice | EurekAlert!
A step closer to future 5G smartphones with the world's first Antenna-on-Display
06.05.2019 | Pohang University of Science & Technology (POSTECH)
The power of randomization: Magnetic skyrmions for novel computer technology
06.05.2019 | Johannes Gutenberg-Universität Mainz
An international collaboration with participation of the University of Bern has demonstrated for the first time in an interference experiment that antimatter particles also behave as waves besides having particle properties. This success paves the way to a new field of investigations of antimatter.
Matter waves constitute a crucial feature of quantum mechanics, where particles have wave properties in addition to particle characteristics. This...
A photodetector converts light into an electrical signal, causing the light to be lost. Researchers led by Tracy Northup at the University of Innsbruck have now built a quantum sensor that can measure light particles non-destructively. It can be used to further investigate the quantum properties of light.
Physicist Tracy Northup is currently researching the development of quantum internet at the University of Innsbruck. The American citizen builds interfaces...
It is only an inconspicuous piece of paper, but it is an important milestone for autonomous driving: At the end of 2018 the three partners from the joint research project RadarGlass applied for a patent for an innovative radar system. The Fraunhofer Institute for Laser Technology ILT from Aachen, the Fraunhofer Institute for Organic Electronics, Electron Beam and Plasma Technology FEP from Dresden and the Institute of High Frequency Technology IHF of RWTH Aachen University have developed a coating process chain that enables radar sensors to be integrated in car headlights. After almost two years in development they have manufactured a working prototype.
Completely autonomous vehicles pose an enormous challenge for sensor technology because, in principle, the supporting system must hear, see and feel better...
Physics and Chemistry scholars from Hong Kong Baptist University (HKBU) have invented a new method which could speed up the drug discovery process and lead to the production of higher quality medicinal drugs which are purer and have no side effects. The technique, which is a world-first breakthrough, uses a specific nanomaterial layer to detect the target molecules in pharmaceuticals and pesticides in just five minutes.
The new HKBU invention can be applied to the drug discovery process, as well as the production and quality control stages of pharmaceutical manufacturing. It...
The use of potassium bromide in the production of graphene on a copper surface can lead to better results. When potassium bromide molecules arrange themselves between graphene and copper, it results in electronic decoupling. This alters the electrical properties of the graphene produced, bringing them closer to pure graphene, as reported by physicists from the universities of Basel, Modena and Munich in the journal ACS Nano.
Graphene consists of a layer of carbon atoms just one atom in thickness in a honeycomb pattern and is the subject of intensive worldwide research.
29.04.2019 | Event News
17.04.2019 | Event News
15.04.2019 | Event News
06.05.2019 | Materials Sciences
06.05.2019 | Life Sciences
06.05.2019 | Information Technology