Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Controlling robots with brainwaves and hand gestures

20.06.2018

System enables people to correct robot mistakes on multi-choice problems

Getting robots to do things isn't easy: usually scientists have to either explicitly program them or get them to understand how humans communicate via language.


By monitoring brain activity, the system can detect in real time if a person notices an error as a robot does a task.

Credit: MIT CSAIL

But what if we could control robots more intuitively, using just hand gestures and brainwaves?

A new system spearheaded by researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) aims to do exactly that, allowing users to instantly correct robot mistakes with nothing more than brain signals and the flick of a finger.

Building off the team's past work focused on simple binary-choice activities, the new work expands the scope to multiple-choice tasks, opening up new possibilities for how human workers could manage teams of robots.

By monitoring brain activity, the system can detect in real time if a person notices an error as a robot does a task. Using an interface that measures muscle activity, the person can then make hand gestures to scroll through and select the correct option for the robot to execute.

Video: https://www.youtube.com/watch?v=_Or8Lt3YtEA&feature=youtu.be

The team demonstrated the system on a task in which a robot moves a power drill to one of three possible targets on the body of a mock plane. Importantly, they showed that the system works on people it's never seen before, meaning that organizations could deploy it in real-world settings without needing to train it on users.

"This work combining EEG and EMG feedback enables natural human-robot interactions for a broader set of applications than we've been able to do before using only EEG feedback," says CSAIL director Daniela Rus, who supervised the work. "By including muscle feedback, we can use gestures to command the robot spatially, with much more nuance and specificity."

PhD candidate Joseph DelPreto was lead author on a paper about the project alongside Rus, former CSAIL postdoctoral associate Andres F. Salazar-Gomez, former CSAIL research scientist Stephanie Gil, research scholar Ramin M. Hasani, and Boston University professor Frank H. Guenther. The paper will be presented at the Robotics: Science and Systems (RSS) conference taking place in Pittsburgh next week.

Intuitive human-robot interaction

In most previous work, systems could generally only recognize brain signals when people trained themselves to "think" in very specific but arbitrary ways and when the system was trained on such signals. For instance, a human operator might have to look at different light displays that correspond to different robot tasks during a training session.

Not surprisingly, such approaches are difficult for people to handle reliably, especially if they work in fields like construction or navigation that already require intense concentration.

Meanwhile, Rus' team harnessed the power of brain signals called "error-related potentials" (ErrPs), which researchers have found to naturally occur when people notice mistakes. If there's an ErrP, the system stops so the user can correct it; if not, it carries on.

"What's great about this approach is that there's no need to train users to think in a prescribed way," says DelPreto. "The machine adapts to you, and not the other way around."

For the project the team used "Baxter", a humanoid robot from Rethink Robotics. With human supervision, the robot went from choosing the correct target 70 percent of the time to more than 97 percent of the time.

To create the system the team harnessed the power of electroencephalography (EEG) for brain activity and electromyography (EMG) for muscle activity, putting a series of electrodes on the users' scalp and forearm.

Both metrics have some individual shortcomings: EEG signals are not always reliably detectable, while EMG signals can sometimes be difficult to map to motions that are any more specific than "move left or right." Merging the two, however, allows for more robust bio-sensing and makes it possible for the system to work on new users without training.

"By looking at both muscle and brain signals, we can start to pick up on a person's natural gestures along with their snap decisions about whether something is going wrong," says DelPreto. "This helps make communicating with a robot more like communicating with another person."

The team says that they could imagine the system one day being useful for the elderly, or workers with language disorders or limited mobility.

"We'd like to move away from a world where people have to adapt to the constraints of machines," says Rus. "Approaches like this show that it's very much possible to develop robotic systems that are a more natural and intuitive extension of us."

###

The project was funded, in part, by the Boeing Company.

Media Contact

Rachel Gordon
rachelg@csail.mit.edu
617-258-0675

 @mit_csail

http://www.csail.mit.edu/ 

Rachel Gordon | EurekAlert!

More articles from Information Technology:

nachricht Multifunctional e-glasses monitor health, protect eyes, control video game
28.05.2020 | American Chemical Society

nachricht Researchers incorporate computer vision and uncertainty into AI for robotic prosthetics
28.05.2020 | North Carolina State University

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Biotechnology: Triggered by light, a novel way to switch on an enzyme

In living cells, enzymes drive biochemical metabolic processes enabling reactions to take place efficiently. It is this very ability which allows them to be used as catalysts in biotechnology, for example to create chemical products such as pharmaceutics. Researchers now identified an enzyme that, when illuminated with blue light, becomes catalytically active and initiates a reaction that was previously unknown in enzymatics. The study was published in "Nature Communications".

Enzymes: they are the central drivers for biochemical metabolic processes in every living cell, enabling reactions to take place efficiently. It is this very...

Im Focus: New double-contrast technique picks up small tumors on MRI

Early detection of tumors is extremely important in treating cancer. A new technique developed by researchers at the University of California, Davis offers a significant advance in using magnetic resonance imaging to pick out even very small tumors from normal tissue. The work is published May 25 in the journal Nature Nanotechnology.

researchers at the University of California, Davis offers a significant advance in using magnetic resonance imaging to pick out even very small tumors from...

Im Focus: I-call - When microimplants communicate with each other / Innovation driver digitization - "Smart Health“

Microelectronics as a key technology enables numerous innovations in the field of intelligent medical technology. The Fraunhofer Institute for Biomedical Engineering IBMT coordinates the BMBF cooperative project "I-call" realizing the first electronic system for ultrasound-based, safe and interference-resistant data transmission between implants in the human body.

When microelectronic systems are used for medical applications, they have to meet high requirements in terms of biocompatibility, reliability, energy...

Im Focus: When predictions of theoretical chemists become reality

Thomas Heine, Professor of Theoretical Chemistry at TU Dresden, together with his team, first predicted a topological 2D polymer in 2019. Only one year later, an international team led by Italian researchers was able to synthesize these materials and experimentally prove their topological properties. For the renowned journal Nature Materials, this was the occasion to invite Thomas Heine to a News and Views article, which was published this week. Under the title "Making 2D Topological Polymers a reality" Prof. Heine describes how his theory became a reality.

Ultrathin materials are extremely interesting as building blocks for next generation nano electronic devices, as it is much easier to make circuits and other...

Im Focus: Rolling into the deep

Scientists took a leukocyte as the blueprint and developed a microrobot that has the size, shape and moving capabilities of a white blood cell. Simulating a blood vessel in a laboratory setting, they succeeded in magnetically navigating the ball-shaped microroller through this dynamic and dense environment. The drug-delivery vehicle withstood the simulated blood flow, pushing the developments in targeted drug delivery a step further: inside the body, there is no better access route to all tissues and organs than the circulatory system. A robot that could actually travel through this finely woven web would revolutionize the minimally-invasive treatment of illnesses.

A team of scientists from the Max Planck Institute for Intelligent Systems (MPI-IS) in Stuttgart invented a tiny microrobot that resembles a white blood cell...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Dresden Nexus Conference 2020: Same Time, Virtual Format, Registration Opened

19.05.2020 | Event News

Aachen Machine Tool Colloquium AWK'21 will take place on June 10 and 11, 2021

07.04.2020 | Event News

International Coral Reef Symposium in Bremen Postponed by a Year

06.04.2020 | Event News

 
Latest News

Biophysicists reveal how optogenetic tool works

29.05.2020 | Life Sciences

Convenient location of a near-threshold proton-emitting resonance in 11B

29.05.2020 | Physics and Astronomy

Mapping immune cells in brain tumors

29.05.2020 | Life Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>