In a paper published online Sept. 7, 2010, in the journal Nature Communications, Thomas Serre and a team of colleagues at the McGovern Institute for Brain Research at MIT and the California Institute of Technology describe a new computer system that is as accurate as people in identifying mouse behaviors in videos.
What’s more, the team is making the fully customizable open-source software available for free. Given standard camcorder footage of a mouse, the software will automatically identify a mouse’s behavior frame by frame.
“We measured the agreement between any two human observers and it was more than 70 percent,” said Serre, who joined the faculty of Brown University in January 2010 after conducting his doctoral and postdoctoral studies, including the work described in the paper, in Tomaso Poggio’s lab at MIT. “The system agreed with humans at the same level. There was no significant difference between the annotations provided by our system and any two human observers.”
The value of the software is not only that it could relieve graduate students and lab technicians from some boredom. It takes about 25 person-hours to fully annotate an hour of mouse movies. In a small experiment with 10 mice who are each observed for 5 hours, that’s 1,250 person-hours of work. Because it is computerized, the system might also provide less subjective annotations than a human team would and could therefore be less susceptible to bias.
“This is a small step towards developing automatic tools for quantifying phenotyping of behavior,” said Poggio. “In the quest to understand the causes of mental diseases, labs at McGovern and elsewhere can rely on precise, quantitative and affordable tools to analyze the genes that contribute to disease. The bottleneck is the lack of corresponding techniques for quantifying the behavioral effects of mental diseases in animal models and in humans. The combination of large-scale genotyping and phenotyping will allow powerful data analysis techniques to help uncover the complex relationship between multiple genes and complex behaviors.”
There are a few commercial programs on the market, some of which cost thousands of dollars. They mostly base their behavioral coding on sensors, rather than video, and therefore have agreement rates with human observers of around 60 percent, substantially lower than the rates between people or between people and the system reported in the paper.
Although feats of artificial perception that compare to real perception are notable, it should not be a surprise that the system matches human levels of observation. It is, after all, based on a computer model of how the human brain interprets what it sees.
“It’s mimicking what the visual system does when you process motion,” Serre says.
In addition, the system learns from experience. To train it to detect grooming behavior, for example, the researchers fed the system lots of videos of mice grooming themselves and certified what the behavior was so the system would know. From there the software was able to identify new scenes of grooming without any coaching. In the paper, the team shows that the software is capable of performing the chore even in different strains of mice in a variety of lighting and other conditions. Serre says the software is likely to be easy to train to work with other lab animals.
“Neuroscience is beginning to give us useful blueprints for a more powerful artificial vision technology,” Poggio explains. “It is encouraging that studies of the brain can lead to a system like ours that can help the scientific community better understand mental diseases.”
The paper’s other authors are Hueihan Jhuang, Estibaliz Garrote, Jim Mutch and Tomaso Poggio at MIT, and Xinlin Yu, Vinita Khilnani, and Andrew D. Steele at Caltech.
Funding: McGovern Institute Neurotechnology (MINT) Program at the McGovern Institute for Brain Research at MIT, Broad Fellows Program in Brain Circuitry at Caltech, and the Taiwan National Science Council.
Julie Pryor | Newswise Science News
Amputees can learn to control a robotic arm with their minds
28.11.2017 | University of Chicago Medical Center
The importance of biodiversity in forests could increase due to climate change
17.11.2017 | Deutsches Zentrum für integrative Biodiversitätsforschung (iDiv) Halle-Jena-Leipzig
Tiny pores at a cell's entryway act as miniature bouncers, letting in some electrically charged atoms--ions--but blocking others. Operating as exquisitely sensitive filters, these "ion channels" play a critical role in biological functions such as muscle contraction and the firing of brain cells.
To rapidly transport the right ions through the cell membrane, the tiny channels rely on a complex interplay between the ions and surrounding molecules,...
The miniaturization of the current technology of storage media is hindered by fundamental limits of quantum mechanics. A new approach consists in using so-called spin-crossover molecules as the smallest possible storage unit. Similar to normal hard drives, these special molecules can save information via their magnetic state. A research team from Kiel University has now managed to successfully place a new class of spin-crossover molecules onto a surface and to improve the molecule’s storage capacity. The storage density of conventional hard drives could therefore theoretically be increased by more than one hundred fold. The study has been published in the scientific journal Nano Letters.
Over the past few years, the building blocks of storage media have gotten ever smaller. But further miniaturization of the current technology is hindered by...
With innovative experiments, researchers at the Helmholtz-Zentrums Geesthacht and the Technical University Hamburg unravel why tiny metallic structures are extremely strong
Light-weight and simultaneously strong – porous metallic nanomaterials promise interesting applications as, for instance, for future aeroplanes with enhanced...
An interdisciplinary group of researchers interfaced individual bacteria with a computer to build a hybrid bio-digital circuit - Study published in Nature Communications
Scientists at the Institute of Science and Technology Austria (IST Austria) have managed to control the behavior of individual bacteria by connecting them to a...
Physicists in the Laboratory for Attosecond Physics (run jointly by LMU Munich and the Max Planck Institute for Quantum Optics) have developed an attosecond electron microscope that allows them to visualize the dispersion of light in time and space, and observe the motions of electrons in atoms.
The most basic of all physical interactions in nature is that between light and matter. This interaction takes place in attosecond times (i.e. billionths of a...
11.12.2017 | Event News
08.12.2017 | Event News
07.12.2017 | Event News
12.12.2017 | Life Sciences
12.12.2017 | Information Technology
12.12.2017 | Ecology, The Environment and Conservation