AI Can See Through Walls – And You Thought Only Superman Could Do This

June 14, 2018 – The Computer Science and Artificial Intelligence Laboratory (CSAIL) at Massachusetts Institute of Technology (MIT) may soon be creeping all of us out with a new application they call RF-Pose. Using AI and wireless signals the application senses people’s movement through walls (see image above).

It uses a neural network which studied radio signals bounced off moving people. It then creates images that move regardless of location whether the person is front of you, or in the next room.

A combination of computer vision, wireless networks, and AI algorithms overcome the limitation that comes in only seeing things using the visible light spectrum. The captured images created from radio frequency waves are projected to a screen. But rather than rendering them in full three-dimension, RF-Pose converts the feedback into stick figures that move, stand or sit.

So how did RF-Pose come to be? 

The research team wanted to create an application that could be useful in places like hospitals, nursing homes, and private residences, to help deal with people suffering from mobility and balance problems, i.e., Parkinson’s Disease, Multiple Sclerosis, Muscular Dystrophy, and a medically frail aging population.

Without a continuous intrusive presence, the idea was to create a tool that an operator could use remotely to see through walls and monitor ambulatory behaviour, and then be able to intercede to prevent falls and injuries while giving greater privacy to those being watched.

The key to making the AI work was the employment of a neural network to analyze the invisible radio spectrum and convert the feedback received into two-dimensional animations that accurately reflected what a person was doing whether visible or behind a wall.

Neural networks have some distinct advantages over other forms of AI. Because they are emulations of our human brain, they are capable of discerning imprecise data and interpreting it. They are great at learning by example. And they don’t take long to figure things out before dispensing with inputs not needed to meet the final objective of the application in which they are used.

In the case of RF-Pose, the neural network needed to learn to understand human movement. To do this the application received input initially from a web camera combined with a wireless sensor so that visual and RF signals were synchronized.

Then the application was exposed to hundreds of different people doing diverse activities in a variety of indoor environments. It watched people walking, sitting, taking stairs, opening doors, waiting for elevators, and more.

As it compiled both a visual and wireless record of movement it soon was able to identify the presence of individuals through walls with 83% accuracy exclusively relying on wireless input.

It soon no longer required human supervision, annotation, or additional examples to demonstrate its capabilities.

The application was able to use a wireless signal 1,000 times lower in strength than that used by WiFi networks ensuring there could be no potential health issues coming from adding additional radio waves to the environments where it would be conceivably used.

In a recently published paper entitled, “Through-Wall Human Pose Estimation Using Radio Signals,” the MIT CSAIL team describes the science, engineering, and benefits that could be derived from the application. Dina Katabi, a professor and team leader in CSAIL, and a co-writer of the paper states in an article published at MIT News, “We’ve seen that monitoring patients’ walking speed and ability to do basic activities on their own gives healthcare providers a window into their lives that they didn’t have before, which could be meaningful for a whole range of diseases. A key advantage of our approach is that patients do not have to wear sensors or remember to charge their devices.”

In addition to healthcare applications, RF-Pose has potential use for search and rescue operations where seeing through walls or rubble would be extremely useful. And of course, one can imagine using the application for all kinds of new and interesting video and virtual reality games.