Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Algorithm enables computers to identify actions much more efficiently

14.05.2014

Techniques from natural-language processing enable computers to efficiently search video for actions

With the commodification of digital cameras, digital video has become so easy to produce that human beings can have trouble keeping up with it. Among the tools that computer scientists are developing to make the profusion of video more useful are algorithms for activity recognition — or determining what the people on camera are doing when.

At the Conference on Computer Vision and Pattern Recognition in June, Hamed Pirsiavash, a postdoc at MIT, and his former thesis advisor, Deva Ramanan of the University of California at Irvine, will present a new activity-recognition algorithm that has several advantages over its predecessors.

One is that the algorithm's execution time scales linearly with the size of the video file it's searching. That means that if one file is 10 times the size of another, the new algorithm will take 10 times as long to search it — not 1,000 times as long, as some earlier algorithms would.

Another is that the algorithm is able to make good guesses about partially completed actions, so it can handle streaming video. Partway through an action, it will issue a probability that the action is of the type that it's looking for. It may revise that probability as the video continues, but it doesn't have to wait until the action is complete to assess it.

Finally, the amount of memory the algorithm requires is fixed, regardless of how many frames of video it's already reviewed. That means that, unlike many of its predecessors, it can handle video streams of any length (or files of any size).

The grammar of action

Enabling all of these advances is the appropriation of a type of algorithm used in natural language processing, the computer science discipline that seeks techniques for interpreting sentences written in natural language.

"One of the challenging problems they try to solve is, if you have a sentence, you want to basically parse the sentence, saying what is the subject, what is the verb, what is the adverb," Pirsiavash says. "We see an analogy here, which is, if you have a complex action — like making tea or making coffee — that has some subactions, we can basically stitch together these subactions and look at each one as something like verb, adjective, and adverb."

On that analogy, the rules defining relationships between subactions are like rules of grammar. When you make tea, for instance, it doesn't matter whether you first put the teabag in the cup or put the kettle on the stove. But it's essential that you put the kettle on the stove before pouring the water into the cup. Similarly, in a given language, it could be the case that nouns can either precede or follow verbs, but that adjectives must always precede nouns.

For any given action, Pirsiavash and Ramanan's algorithm must thus learn a new "grammar." And the mechanism that it uses is the one that many natural-language-processing systems rely on: machine learning. Pirsiavash and Ramanan feed their algorithm training examples of videos depicting a particular action, and specify the number of subactions that the algorithm should look for. But they don't give it any information about what those subactions are, or what the transitions between them look like.

Pruning possibilities

The rules relating subactions are the key to the algorithm's efficiency. As a video plays, the algorithm constructs a set of hypotheses about which subactions are being depicted where, and it ranks them according to probability. It can't limit itself to a single hypothesis, as each new frame could require it to revise its probabilities. But it can eliminate hypotheses that don't conform to its grammatical rules, which dramatically limits the number of possibilities it has to canvass.

The researchers tested their algorithm on eight different types of athletic endeavor — such as weightlifting and bowling — with training videos culled from YouTube. They found that, according to metrics standard in the field of computer vision, their algorithm identified new instances of the same activities more accurately than its predecessors.

Pirsiavash is particularly interested in possible medical applications of action detection. The proper execution of physical-therapy exercises, for instance, could have a grammar that's distinct from improper execution; similarly, the return of motor function in patients with neurological damage could be identified by its unique grammar. Action-detection algorithms could also help determine whether, for instance, elderly patients remembered to take their medication — and issue alerts if they didn't.

Abby Abazorius | newswise
Further information:
http://www.mit.edu

Further reports about: Massachusetts Technology algorithm identified identify problems

More articles from Information Technology:

nachricht Computing at the Speed of Light
22.05.2015 | University of Utah

nachricht NOAA's GOES-R satellite begins environmental testing
22.05.2015 | NASA/Goddard Space Flight Center

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Advance in regenerative medicine

The only professorship in Germany to date, one master's programme, one laboratory with worldwide unique equipment and the corresponding research results: The University of Würzburg is leading in the field of biofabrication.

Paul Dalton is presently the only professor of biofabrication in Germany. About a year ago, the Australian researcher relocated to the Würzburg department for...

Im Focus: Basel Physicists Develop Efficient Method of Signal Transmission from Nanocomponents

Physicists have developed an innovative method that could enable the efficient use of nanocomponents in electronic circuits. To achieve this, they have developed a layout in which a nanocomponent is connected to two electrical conductors, which uncouple the electrical signal in a highly efficient manner. The scientists at the Department of Physics and the Swiss Nanoscience Institute at the University of Basel have published their results in the scientific journal “Nature Communications” together with their colleagues from ETH Zurich.

Electronic components are becoming smaller and smaller. Components measuring just a few nanometers – the size of around ten atoms – are already being produced...

Im Focus: IoT-based Advanced Automobile Parking Navigation System

Development and implementation of an advanced automobile parking navigation platform for parking services

To fulfill the requirements of the industry, PolyU researchers developed the Advanced Automobile Parking Navigation Platform, which includes smart devices,...

Im Focus: First electrical car ferry in the world in operation in Norway now

  • Siemens delivers electric propulsion system and charging stations with lithium-ion batteries charged from hydro power
  • Ferry only uses 150 kilowatt hours (kWh) per route and reduces cost of fuel by 60 percent
  • Milestone on the road to operating emission-free ferries

The world's first electrical car and passenger ferry powered by batteries has entered service in Norway. The ferry only uses 150 kWh per route, which...

Im Focus: Into the ice – RV Polarstern opens the arctic season by setting course for Spitsbergen

On Tuesday, 19 May 2015 the research icebreaker Polarstern will leave its home port in Bremerhaven, setting a course for the Arctic. Led by Dr Ilka Peeken from the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research (AWI) a team of 53 researchers from 11 countries will investigate the effects of climate change in the Arctic, from the surface ice floes down to the seafloor.

RV Polarstern will enter the sea-ice zone north of Spitsbergen. Covering two shallow regions on their way to deeper waters, the scientists on board will focus...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

International symposium: trends in spatial analysis and modelling for a more sustainable land use

20.05.2015 | Event News

15th conference of the International Association of Colloid and Interface Scientists

18.05.2015 | Event News

EHFG 2015: Securing health in Europe. Balancing priorities, sharing responsibilities

12.05.2015 | Event News

 
Latest News

Advance in regenerative medicine

27.05.2015 | Health and Medicine

Analytical lamps monitor air pollution in cities

26.05.2015 | Ecology, The Environment and Conservation

DNA double helix does double duty in assembling arrays of nanoparticles

26.05.2015 | Life Sciences

VideoLinks
B2B-VideoLinks
More VideoLinks >>>