Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

AI senses people's pose through walls

12.06.2018

X-ray vision has long seemed like a far-fetched sci-fi fantasy, but over the last decade a team led by Professor Dina Katabi from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) has continually gotten us closer to seeing through walls.

Their latest project, "RF-Pose," uses artificial intelligence (AI) to teach wireless devices to sense people's postures and movement, even from the other side of a wall.


This is Mingmin Zhao, pictured right.

Credit: Jason Dorfman, MIT CSAIL

The researchers use a neural network to analyze radio signals that bounce off people's bodies, and can then create a dynamic stick figure that walks, stops, sits and moves its limbs as the person performs those actions.

The team says that the system could be used to monitor diseases like Parkinson's and multiple sclerosis (MS), providing a better understanding of disease progression and allowing doctors to adjust medications accordingly. It could also help elderly people live more independently, while providing the added security of monitoring for falls, injuries and changes in activity patterns.

(All data the team collected has subjects' consent and is anonymized and encrypted to protect user privacy. For future real-world applications, the team plans to implement a "consent mechanism" in which the person who installs the device is cued to do a specific set of movements in order for it to begin to monitor the environment.)

The team is currently working with doctors to explore multiple applications in healthcare.

"We've seen that monitoring patients' walking speed and ability to do basic activities on their own gives healthcare providers a window into their lives that they didn't have before, which could be meaningful for a whole range of diseases," says Katabi, who co-wrote a new paper about the project. "A key advantage of our approach is that patients do not have to wear sensors or remember to charge their devices."

Besides health-care, the team says that RF-Pose could also be used for new classes of video games where players move around the house, or even in search-and-rescue missions to help locate survivors.

"Just like how cellphones and Wi-Fi routers have become essential parts of today's households, I believe that wireless technologies like these will help power the homes of the future," says Katabi, who co-wrote the new paper with PhD student and lead author Mingmin Zhao, MIT professor Antonio Torralba, postdoc Mohammad Abu Alsheikh, graduate student Tianhong Li and PhD students Yonglong Tian and Hang Zhao. They will present it later this month at the Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City, Utah.

One challenge the researchers had to address is that most neural networks are trained using data labeled by hand. A neural network trained to identify cats, for example, requires that people look at a big dataset of images and label each one as either "cat" or "not cat." Radio signals, meanwhile, can't be easily labeled by humans.

To address this, the researchers collected examples using both their wireless device and a camera. They gathered thousands of images of people doing activities like walking, talking, sitting, opening doors and waiting for elevators.

They then used these images from the camera to extract the stick figures, which they showed to the neural network along with the corresponding radio signal. This combination of examples enabled the system to learn the association between the radio signal and the stick figures of the people in the scene.

Post-training, RF-Pose was able to estimate a person's posture and movements without cameras, using only the wireless reflections that bounce off people's bodies.

Since cameras can't see through walls, the network was never explicitly trained on data from the other side of a wall - which is what made it particularly surprising to the MIT team that the network could generalize its knowledge to be able to handle through-wall movement.

"If you think of the computer vision system as the teacher, this is a truly fascinating example of the student outperforming the teacher," says Torralba.

Besides sensing movement, the authors also showed that they could use wireless signals to accurately identify somebody 83 percent of the time out of a line-up of 100 individuals. This ability could be particularly useful for the application of search-and-rescue operations, when it may be helpful to know the identity of specific people.

For this paper, the model outputs a 2-D stick figure, but the team is also working to create 3-D representations that would be able to reflect even smaller micromovements. For example, it might be able to see if an older person's hands are shaking regularly enough that they may want to get a check-up.

"By using this combination of visual data and AI to see through walls, we can enable better scene understanding and smarter environments to live safer, more productive lives," says Zhao.

Media Contact

Adam Conner-Simons
aconner@csail.mit.edu
617-324-9135

 @mit_csail

http://www.csail.mit.edu/ 

Adam Conner-Simons | EurekAlert!

Further reports about: CSAIL MIT cameras movements neural network radio signal

More articles from Information Technology:

nachricht Advanced AI boosts clinical analysis of eye images
19.09.2019 | Universitätsspital Bern

nachricht Quantum computers by AQT and University of Innsbruck leverage Cirq for quantum algorithm development
16.09.2019 | Universität Innsbruck

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: 'Nanochains' could increase battery capacity, cut charging time

How long the battery of your phone or computer lasts depends on how many lithium ions can be stored in the battery's negative electrode material. If the battery runs out of these ions, it can't generate an electrical current to run a device and ultimately fails.

Materials with a higher lithium ion storage capacity are either too heavy or the wrong shape to replace graphite, the electrode material currently used in...

Im Focus: Stevens team closes in on 'holy grail' of room temperature quantum computing chips

Photons interact on chip-based system with unprecedented efficiency

To process information, photons must interact. However, these tiny packets of light want nothing to do with each other, each passing by without altering the...

Im Focus: Happy hour for time-resolved crystallography

Researchers from the Department of Atomically Resolved Dynamics of the Max Planck Institute for the Structure and Dynamics of Matter (MPSD) at the Center for Free-Electron Laser Science in Hamburg, the University of Hamburg and the European Molecular Biology Laboratory (EMBL) outstation in the city have developed a new method to watch biomolecules at work. This method dramatically simplifies starting enzymatic reactions by mixing a cocktail of small amounts of liquids with protein crystals. Determination of the protein structures at different times after mixing can be assembled into a time-lapse sequence that shows the molecular foundations of biology.

The functions of biomolecules are determined by their motions and structural changes. Yet it is a formidable challenge to understand these dynamic motions.

Im Focus: Modular OLED light strips

At the International Symposium on Automotive Lighting 2019 (ISAL) in Darmstadt from September 23 to 25, 2019, the Fraunhofer Institute for Organic Electronics, Electron Beam and Plasma Technology FEP, a provider of research and development services in the field of organic electronics, will present OLED light strips of any length with additional functionalities for the first time at booth no. 37.

Almost everyone is familiar with light strips for interior design. LED strips are available by the metre in DIY stores around the corner and are just as often...

Im Focus: Tomorrow´s coolants of choice

Scientists assess the potential of magnetic-cooling materials

Later during this century, around 2060, a paradigm shift in global energy consumption is expected: we will spend more energy for cooling than for heating....

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Optical Technologies: International Symposium „Future Optics“ in Hannover

19.09.2019 | Event News

Society 5.0: putting humans at the heart of digitalisation

10.09.2019 | Event News

Interspeech 2019 conference: Alexa and Siri in Graz

04.09.2019 | Event News

 
Latest News

Quality control in immune communication: Chaperones detect immature signaling molecules in the immune system

20.09.2019 | Life Sciences

Moderately Common Plants Show Highest Relative Losses

20.09.2019 | Life Sciences

The Fluid Fingerprint of Hurricanes

20.09.2019 | Life Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>