Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Robot can be programmed by casually talking to it

23.06.2014

Robots are getting smarter, but they still need step-by-step instructions for tasks they haven't performed before.

Before you can tell your household robot "Make me a bowl of ramen noodles," you'll have to teach it how to do that. Since we're not all computer programmers, we'd prefer to give those instructions in English, just as we'd lay out a task for a child.

But human language can be ambiguous, and some instructors forget to mention important details. Suppose you told your household robot how to prepare ramen noodles, but forgot to mention heating the water or tell it where the stove is.

In his Robot Learning Lab, Ashutosh Saxena, assistant professor of computer science at Cornell University, is teaching robots to understand instructions in natural language from various speakers, account for missing information, and adapt to the environment at hand.

... more about:
»Robotics »Science »combining

Saxena and graduate students Dipendra K. Misra and Jaeyong Sung will describe their methods at the Robotics: Science and Systems conference at the University of California, Berkeley, July 12-16.

Video and abstract available at http://tellmedave.cs.cornell.edu

The robot may have a built-in programming language with commands like find (pan); grasp (pan); carry (pan, water tap); fill (pan, water); carry (pan, stove) and so on. Saxena's software translates human sentences, such as "Fill a pan with water, put it on the stove, heat the water. When it's boiling, add the noodles." into robot language. Notice that you didn't say, "Turn on the stove." The robot has to be smart enough to fill in that missing step.

Saxena's robot, equipped with a 3-D camera, scans its environment and identifies the objects in it, using computer vision software previously developed in Saxena's lab. The robot has been trained to associate objects with their capabilities: A pan can be poured into or poured from; stoves can have other objects set on them, and can heat things.

So the robot can identify the pan, locate the water faucet and stove and incorporate that information into its procedure. If you tell it to "heat water" it can use the stove or the microwave, depending on which is available. And it can carry out the same actions tomorrow if you've moved the pan, or even moved the robot to a different kitchen.

Other workers have attacked these problems by giving a robot a set of templates for common actions and chewing up sentences one word at a time. Saxena's research group uses techniques computer scientists call "machine learning" to train the robot's computer brain to associate entire commands with flexibly defined actions. The computer is fed animated video simulations of the action –- created by humans in a process similar to playing a video game – accompanied by recorded voice commands from several different speakers.

The computer stores the combination of many similar commands as a flexible pattern that can match many variations, so when it hears "Take the pot to the stove," "Carry the pot to the stove," "Put the pot on the stove," "Go to the stove and heat the pot" and so on, it calculates the probability of a match with what it has heard before, and if the probability is high enough, it declares a match. A similarly fuzzy version of the video simulation supplies a plan for the action: Wherever the sink and the stove are, the path can be matched to the recorded action of carrying the pot of water from one to the other.

Of course the robot still doesn't get it right all the time. To test, the researchers gave instructions for preparing ramen noodles and for making affogato – an Italian dessert combining coffee and ice cream: "Take some coffee in a cup. Add ice cream of your choice. Finally, add raspberry syrup to the mixture."

The robot performed correctly up to 64 percent of the time even when the commands were varied or the environment was changed, and it was able to fill in missing steps. That was three to four times better than previous methods, the researchers reported, but "There is still room for improvement."

You can teach a simulated robot to perform a kitchen task at the "Tell me Dave" website, and your input there will become part of a crowdsourced library of instructions for the Cornell robots. Aditya Jami, visiting researcher at Cornell, is helping Tell Me Dave to scale the library to millions of examples. "With crowdsourcing at such a scale, robots will learn at a much faster rate," Saxena said.

###

Cornell University has television, ISDN and dedicated Skype/Google+ Hangout studios available for media interviews.

Syl Kacapyr | Eurek Alert!

Further reports about: Robotics Science combining

More articles from Power and Electrical Engineering:

nachricht Scientists develop a metamaterial for applications in magnonics
22.08.2019 | Moscow Institute of Physics and Technology

nachricht Materials that can revolutionize how light is harnessed for solar energy
20.08.2019 | Columbia University

All articles from Power and Electrical Engineering >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Hamburg and Kiel researchers observe spontaneous occurrence of skyrmions in atomically thin cobalt films

Since their experimental discovery, magnetic skyrmions - tiny magnetic knots - have moved into the focus of research. Scientists from Hamburg and Kiel have now been able to show that individual magnetic skyrmions with a diameter of only a few nanometres can be stabilised in magnetic metal films even without an external magnetic field. They report on their discovery in the journal Nature Communications.

The existence of magnetic skyrmions as particle-like objects was predicted 30 years ago by theoretical physicists, but could only be proven experimentally in...

Im Focus: Physicists create world's smallest engine

Theoretical physicists at Trinity College Dublin are among an international collaboration that has built the world's smallest engine - which, as a single calcium ion, is approximately ten billion times smaller than a car engine.

Work performed by Professor John Goold's QuSys group in Trinity's School of Physics describes the science behind this tiny motor.

Im Focus: Quantum computers to become portable

Together with the University of Innsbruck, the ETH Zurich and Interactive Fully Electrical Vehicles SRL, Infineon Austria is researching specific questions on the commercial use of quantum computers. With new innovations in design and manufacturing, the partners from universities and industry want to develop affordable components for quantum computers.

Ion traps have proven to be a very successful technology for the control and manipulation of quantum particles. Today, they form the heart of the first...

Im Focus: Towards an 'orrery' for quantum gauge theory

Experimental progress towards engineering quantized gauge fields coupled to ultracold matter promises a versatile platform to tackle problems ranging from condensed-matter to high-energy physics

The interaction between fields and matter is a recurring theme throughout physics. Classical cases such as the trajectories of one celestial body moving in the...

Im Focus: A miniature stretchable pump for the next generation of soft robots

Soft robots have a distinct advantage over their rigid forebears: they can adapt to complex environments, handle fragile objects and interact safely with humans. Made from silicone, rubber or other stretchable polymers, they are ideal for use in rehabilitation exoskeletons and robotic clothing. Soft bio-inspired robots could one day be deployed to explore remote or dangerous environments.

Most soft robots are actuated by rigid, noisy pumps that push fluids into the machines' moving parts. Because they are connected to these bulky pumps by tubes,...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

The power of thought – the key to success: CYBATHLON BCI Series 2019

16.08.2019 | Event News

4th Hybrid Materials and Structures 2020 28 - 29 April 2020, Karlsruhe, Germany

14.08.2019 | Event News

What will the digital city of the future look like? City Science Summit on 1st and 2nd October 2019 in Hamburg

12.08.2019 | Event News

 
Latest News

Building an atomic-scale vacuum trap for spin-polarized electrons

23.08.2019 | Physics and Astronomy

Novel paradigm in drug development: Understanding resistance mechanisms to targeted protein degradation

23.08.2019 | Life Sciences

Hamburg and Kiel researchers observe spontaneous occurrence of skyrmions in atomically thin cobalt films

23.08.2019 | Physics and Astronomy

VideoLinks
Science & Research
Overview of more VideoLinks >>>