Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

AI senses people's pose through walls

12.06.2018

X-ray vision has long seemed like a far-fetched sci-fi fantasy, but over the last decade a team led by Professor Dina Katabi from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) has continually gotten us closer to seeing through walls.

Their latest project, "RF-Pose," uses artificial intelligence (AI) to teach wireless devices to sense people's postures and movement, even from the other side of a wall.


This is Mingmin Zhao, pictured right.

Credit: Jason Dorfman, MIT CSAIL

The researchers use a neural network to analyze radio signals that bounce off people's bodies, and can then create a dynamic stick figure that walks, stops, sits and moves its limbs as the person performs those actions.

The team says that the system could be used to monitor diseases like Parkinson's and multiple sclerosis (MS), providing a better understanding of disease progression and allowing doctors to adjust medications accordingly. It could also help elderly people live more independently, while providing the added security of monitoring for falls, injuries and changes in activity patterns.

(All data the team collected has subjects' consent and is anonymized and encrypted to protect user privacy. For future real-world applications, the team plans to implement a "consent mechanism" in which the person who installs the device is cued to do a specific set of movements in order for it to begin to monitor the environment.)

The team is currently working with doctors to explore multiple applications in healthcare.

"We've seen that monitoring patients' walking speed and ability to do basic activities on their own gives healthcare providers a window into their lives that they didn't have before, which could be meaningful for a whole range of diseases," says Katabi, who co-wrote a new paper about the project. "A key advantage of our approach is that patients do not have to wear sensors or remember to charge their devices."

Besides health-care, the team says that RF-Pose could also be used for new classes of video games where players move around the house, or even in search-and-rescue missions to help locate survivors.

"Just like how cellphones and Wi-Fi routers have become essential parts of today's households, I believe that wireless technologies like these will help power the homes of the future," says Katabi, who co-wrote the new paper with PhD student and lead author Mingmin Zhao, MIT professor Antonio Torralba, postdoc Mohammad Abu Alsheikh, graduate student Tianhong Li and PhD students Yonglong Tian and Hang Zhao. They will present it later this month at the Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City, Utah.

One challenge the researchers had to address is that most neural networks are trained using data labeled by hand. A neural network trained to identify cats, for example, requires that people look at a big dataset of images and label each one as either "cat" or "not cat." Radio signals, meanwhile, can't be easily labeled by humans.

To address this, the researchers collected examples using both their wireless device and a camera. They gathered thousands of images of people doing activities like walking, talking, sitting, opening doors and waiting for elevators.

They then used these images from the camera to extract the stick figures, which they showed to the neural network along with the corresponding radio signal. This combination of examples enabled the system to learn the association between the radio signal and the stick figures of the people in the scene.

Post-training, RF-Pose was able to estimate a person's posture and movements without cameras, using only the wireless reflections that bounce off people's bodies.

Since cameras can't see through walls, the network was never explicitly trained on data from the other side of a wall - which is what made it particularly surprising to the MIT team that the network could generalize its knowledge to be able to handle through-wall movement.

"If you think of the computer vision system as the teacher, this is a truly fascinating example of the student outperforming the teacher," says Torralba.

Besides sensing movement, the authors also showed that they could use wireless signals to accurately identify somebody 83 percent of the time out of a line-up of 100 individuals. This ability could be particularly useful for the application of search-and-rescue operations, when it may be helpful to know the identity of specific people.

For this paper, the model outputs a 2-D stick figure, but the team is also working to create 3-D representations that would be able to reflect even smaller micromovements. For example, it might be able to see if an older person's hands are shaking regularly enough that they may want to get a check-up.

"By using this combination of visual data and AI to see through walls, we can enable better scene understanding and smarter environments to live safer, more productive lives," says Zhao.

Media Contact

Adam Conner-Simons
aconner@csail.mit.edu
617-324-9135

 @mit_csail

http://www.csail.mit.edu/ 

Adam Conner-Simons | EurekAlert!

Further reports about: CSAIL MIT cameras movements neural network radio signal

More articles from Information Technology:

nachricht Accelerating quantum technologies with materials processing at the atomic scale
15.05.2019 | University of Oxford

nachricht A step towards probabilistic computing
15.05.2019 | University of Konstanz

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Self-repairing batteries

UTokyo engineers develop a way to create high-capacity long-life batteries

Engineers at the University of Tokyo continually pioneer new ways to improve battery technology. Professor Atsuo Yamada and his team recently developed a...

Im Focus: Quantum Cloud Computing with Self-Check

With a quantum coprocessor in the cloud, physicists from Innsbruck, Austria, open the door to the simulation of previously unsolvable problems in chemistry, materials research or high-energy physics. The research groups led by Rainer Blatt and Peter Zoller report in the journal Nature how they simulated particle physics phenomena on 20 quantum bits and how the quantum simulator self-verified the result for the first time.

Many scientists are currently working on investigating how quantum advantage can be exploited on hardware already available today. Three years ago, physicists...

Im Focus: Accelerating quantum technologies with materials processing at the atomic scale

'Quantum technologies' utilise the unique phenomena of quantum superposition and entanglement to encode and process information, with potentially profound benefits to a wide range of information technologies from communications to sensing and computing.

However a major challenge in developing these technologies is that the quantum phenomena are very fragile, and only a handful of physical systems have been...

Im Focus: A step towards probabilistic computing

Working group led by physicist Professor Ulrich Nowak at the University of Konstanz, in collaboration with a team of physicists from Johannes Gutenberg University Mainz, demonstrates how skyrmions can be used for the computer concepts of the future

When it comes to performing a calculation destined to arrive at an exact result, humans are hopelessly inferior to the computer. In other areas, humans are...

Im Focus: Recording embryonic development

Scientists develop a molecular recording tool that enables in vivo lineage tracing of embryonic cells

The beginning of new life starts with a fascinating process: A single cell gives rise to progenitor cells that eventually differentiate into the three germ...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

SEMANTiCS 2019 brings together industry leaders and data scientists in Karlsruhe

29.04.2019 | Event News

Revered mathematicians and computer scientists converge with 200 young researchers in Heidelberg!

17.04.2019 | Event News

First dust conference in the Central Asian part of the earth’s dust belt

15.04.2019 | Event News

 
Latest News

Summit charts a course to uncover the origins of genetic diseases

22.05.2019 | Life Sciences

New study finds distinct microbes living next to corals

22.05.2019 | Life Sciences

Stellar waltz with dramatic ending

22.05.2019 | Physics and Astronomy

VideoLinks
Science & Research
Overview of more VideoLinks >>>