In a paper published online Sept. 7, 2010, in the journal Nature Communications, Thomas Serre and a team of colleagues at the McGovern Institute for Brain Research at MIT and the California Institute of Technology describe a new computer system that is as accurate as people in identifying mouse behaviors in videos.
What’s more, the team is making the fully customizable open-source software available for free. Given standard camcorder footage of a mouse, the software will automatically identify a mouse’s behavior frame by frame.
“We measured the agreement between any two human observers and it was more than 70 percent,” said Serre, who joined the faculty of Brown University in January 2010 after conducting his doctoral and postdoctoral studies, including the work described in the paper, in Tomaso Poggio’s lab at MIT. “The system agreed with humans at the same level. There was no significant difference between the annotations provided by our system and any two human observers.”
The value of the software is not only that it could relieve graduate students and lab technicians from some boredom. It takes about 25 person-hours to fully annotate an hour of mouse movies. In a small experiment with 10 mice who are each observed for 5 hours, that’s 1,250 person-hours of work. Because it is computerized, the system might also provide less subjective annotations than a human team would and could therefore be less susceptible to bias.
“This is a small step towards developing automatic tools for quantifying phenotyping of behavior,” said Poggio. “In the quest to understand the causes of mental diseases, labs at McGovern and elsewhere can rely on precise, quantitative and affordable tools to analyze the genes that contribute to disease. The bottleneck is the lack of corresponding techniques for quantifying the behavioral effects of mental diseases in animal models and in humans. The combination of large-scale genotyping and phenotyping will allow powerful data analysis techniques to help uncover the complex relationship between multiple genes and complex behaviors.”
There are a few commercial programs on the market, some of which cost thousands of dollars. They mostly base their behavioral coding on sensors, rather than video, and therefore have agreement rates with human observers of around 60 percent, substantially lower than the rates between people or between people and the system reported in the paper.
Although feats of artificial perception that compare to real perception are notable, it should not be a surprise that the system matches human levels of observation. It is, after all, based on a computer model of how the human brain interprets what it sees.
“It’s mimicking what the visual system does when you process motion,” Serre says.
In addition, the system learns from experience. To train it to detect grooming behavior, for example, the researchers fed the system lots of videos of mice grooming themselves and certified what the behavior was so the system would know. From there the software was able to identify new scenes of grooming without any coaching. In the paper, the team shows that the software is capable of performing the chore even in different strains of mice in a variety of lighting and other conditions. Serre says the software is likely to be easy to train to work with other lab animals.
“Neuroscience is beginning to give us useful blueprints for a more powerful artificial vision technology,” Poggio explains. “It is encouraging that studies of the brain can lead to a system like ours that can help the scientific community better understand mental diseases.”
The paper’s other authors are Hueihan Jhuang, Estibaliz Garrote, Jim Mutch and Tomaso Poggio at MIT, and Xinlin Yu, Vinita Khilnani, and Andrew D. Steele at Caltech.
Funding: McGovern Institute Neurotechnology (MINT) Program at the McGovern Institute for Brain Research at MIT, Broad Fellows Program in Brain Circuitry at Caltech, and the Taiwan National Science Council.
Julie Pryor | Newswise Science News
The Great Unknown: Risk-Taking Behavior in Adolescents
19.01.2017 | Max-Planck-Institut für Bildungsforschung
A sudden drop in outdoor temperature increases the risk of respiratory infections
11.01.2017 | University of Gothenburg
An important step towards a completely new experimental access to quantum physics has been made at University of Konstanz. The team of scientists headed by...
Yersiniae cause severe intestinal infections. Studies using Yersinia pseudotuberculosis as a model organism aim to elucidate the infection mechanisms of these...
Researchers from the University of Hamburg in Germany, in collaboration with colleagues from the University of Aarhus in Denmark, have synthesized a new superconducting material by growing a few layers of an antiferromagnetic transition-metal chalcogenide on a bismuth-based topological insulator, both being non-superconducting materials.
While superconductivity and magnetism are generally believed to be mutually exclusive, surprisingly, in this new material, superconducting correlations...
Laser-driving of semimetals allows creating novel quasiparticle states within condensed matter systems and switching between different states on ultrafast time scales
Studying properties of fundamental particles in condensed matter systems is a promising approach to quantum field theory. Quasiparticles offer the opportunity...
Among the general public, solar thermal energy is currently associated with dark blue, rectangular collectors on building roofs. Technologies are needed for aesthetically high quality architecture which offer the architect more room for manoeuvre when it comes to low- and plus-energy buildings. With the “ArKol” project, researchers at Fraunhofer ISE together with partners are currently developing two façade collectors for solar thermal energy generation, which permit a high degree of design flexibility: a strip collector for opaque façade sections and a solar thermal blind for transparent sections. The current state of the two developments will be presented at the BAU 2017 trade fair.
As part of the “ArKol – development of architecturally highly integrated façade collectors with heat pipes” project, Fraunhofer ISE together with its partners...
19.01.2017 | Event News
10.01.2017 | Event News
09.01.2017 | Event News
20.01.2017 | Awards Funding
20.01.2017 | Materials Sciences
20.01.2017 | Life Sciences