In a paper published online Sept. 7, 2010, in the journal Nature Communications, Thomas Serre and a team of colleagues at the McGovern Institute for Brain Research at MIT and the California Institute of Technology describe a new computer system that is as accurate as people in identifying mouse behaviors in videos.
What’s more, the team is making the fully customizable open-source software available for free. Given standard camcorder footage of a mouse, the software will automatically identify a mouse’s behavior frame by frame.
“We measured the agreement between any two human observers and it was more than 70 percent,” said Serre, who joined the faculty of Brown University in January 2010 after conducting his doctoral and postdoctoral studies, including the work described in the paper, in Tomaso Poggio’s lab at MIT. “The system agreed with humans at the same level. There was no significant difference between the annotations provided by our system and any two human observers.”
The value of the software is not only that it could relieve graduate students and lab technicians from some boredom. It takes about 25 person-hours to fully annotate an hour of mouse movies. In a small experiment with 10 mice who are each observed for 5 hours, that’s 1,250 person-hours of work. Because it is computerized, the system might also provide less subjective annotations than a human team would and could therefore be less susceptible to bias.
“This is a small step towards developing automatic tools for quantifying phenotyping of behavior,” said Poggio. “In the quest to understand the causes of mental diseases, labs at McGovern and elsewhere can rely on precise, quantitative and affordable tools to analyze the genes that contribute to disease. The bottleneck is the lack of corresponding techniques for quantifying the behavioral effects of mental diseases in animal models and in humans. The combination of large-scale genotyping and phenotyping will allow powerful data analysis techniques to help uncover the complex relationship between multiple genes and complex behaviors.”
There are a few commercial programs on the market, some of which cost thousands of dollars. They mostly base their behavioral coding on sensors, rather than video, and therefore have agreement rates with human observers of around 60 percent, substantially lower than the rates between people or between people and the system reported in the paper.
Although feats of artificial perception that compare to real perception are notable, it should not be a surprise that the system matches human levels of observation. It is, after all, based on a computer model of how the human brain interprets what it sees.
“It’s mimicking what the visual system does when you process motion,” Serre says.
In addition, the system learns from experience. To train it to detect grooming behavior, for example, the researchers fed the system lots of videos of mice grooming themselves and certified what the behavior was so the system would know. From there the software was able to identify new scenes of grooming without any coaching. In the paper, the team shows that the software is capable of performing the chore even in different strains of mice in a variety of lighting and other conditions. Serre says the software is likely to be easy to train to work with other lab animals.
“Neuroscience is beginning to give us useful blueprints for a more powerful artificial vision technology,” Poggio explains. “It is encouraging that studies of the brain can lead to a system like ours that can help the scientific community better understand mental diseases.”
The paper’s other authors are Hueihan Jhuang, Estibaliz Garrote, Jim Mutch and Tomaso Poggio at MIT, and Xinlin Yu, Vinita Khilnani, and Andrew D. Steele at Caltech.
Funding: McGovern Institute Neurotechnology (MINT) Program at the McGovern Institute for Brain Research at MIT, Broad Fellows Program in Brain Circuitry at Caltech, and the Taiwan National Science Council.
Julie Pryor | Newswise Science News
Win-win strategies for climate and food security
02.10.2017 | International Institute for Applied Systems Analysis (IIASA)
The personality factor: How to foster the sharing of research data
06.09.2017 | ZBW – Leibniz-Informationszentrum Wirtschaft
University of Maryland researchers contribute to historic detection of gravitational waves and light created by event
On August 17, 2017, at 12:41:04 UTC, scientists made the first direct observation of a merger between two neutron stars--the dense, collapsed cores that remain...
Seven new papers describe the first-ever detection of light from a gravitational wave source. The event, caused by two neutron stars colliding and merging together, was dubbed GW170817 because it sent ripples through space-time that reached Earth on 2017 August 17. Around the world, hundreds of excited astronomers mobilized quickly and were able to observe the event using numerous telescopes, providing a wealth of new data.
Previous detections of gravitational waves have all involved the merger of two black holes, a feat that won the 2017 Nobel Prize in Physics earlier this month....
Material defects in end products can quickly result in failures in many areas of industry, and have a massive impact on the safe use of their products. This is why, in the field of quality assurance, intelligent, nondestructive sensor systems play a key role. They allow testing components and parts in a rapid and cost-efficient manner without destroying the actual product or changing its surface. Experts from the Fraunhofer IZFP in Saarbrücken will be presenting two exhibits at the Blechexpo in Stuttgart from 7–10 November 2017 that allow fast, reliable, and automated characterization of materials and detection of defects (Hall 5, Booth 5306).
When quality testing uses time-consuming destructive test methods, it can result in enormous costs due to damaging or destroying the products. And given that...
Using a new cooling technique MPQ scientists succeed at observing collisions in a dense beam of cold and slow dipolar molecules.
How do chemical reactions proceed at extremely low temperatures? The answer requires the investigation of molecular samples that are cold, dense, and slow at...
Scientists from the Max Planck Institute of Quantum Optics, using high precision laser spectroscopy of atomic hydrogen, confirm the surprisingly small value of the proton radius determined from muonic hydrogen.
It was one of the breakthroughs of the year 2010: Laser spectroscopy of muonic hydrogen resulted in a value for the proton charge radius that was significantly...
17.10.2017 | Event News
10.10.2017 | Event News
10.10.2017 | Event News
18.10.2017 | Materials Sciences
18.10.2017 | Physics and Astronomy
18.10.2017 | Physics and Astronomy