Pinpointing Human Activity in a Video Barrage

Video cameras are used to keep an eye on many indoor and outdoor locations, but to pinpoint suspicious activity, human security guards or intelligence analysts have the unenviable task of watching dozens of video monitors or many hours of recorded video.

Supported by an NSF award, Jezekiel Ben-Arie and his students at the University of Illinois at Chicago have developed a technique, much faster and more reliable than previous methods, that allows a computer to recognize a human action contained in a video.

Ben-Arie envisions a system that will allow human analysts to search vast databases of digital video by animating the desired action with virtual “puppets” or by videotaping a person making the movements. Ben-Arie’s method makes it feasible to quickly find matching human motions within large amounts of video. This includes searches through databases of surveillance videos for suspicious activities, such as a person putting down an object and leaving.

“Security guards have to monitor 10 or 20 screens continuously, and it’s very boring,” said Ben-Arie, a professor of electrical and computer engineering. “A machine won’t get bored. It’s much more practical to have the computer do it.”

Ben-Arie’s method, for which he has a provisional patent, identifies and tracks nine major body parts and needs only a few poses from a video segment to distinguish an activity. Yet the method is robust enough to differentiate between activities as similar as walking and running, even with several people in the video.

The technique may have medical applications in physical therapy or analysis of motor function. Other applications might include choreography or sports training, in which a dancer’s or athlete’s movements can be compared against ideal or standardized movements. Ben-Arie and his students described the method recently in IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)