Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Robot can be programmed by casually talking to it

23.06.2014

Robots are getting smarter, but they still need step-by-step instructions for tasks they haven't performed before.

Before you can tell your household robot "Make me a bowl of ramen noodles," you'll have to teach it how to do that. Since we're not all computer programmers, we'd prefer to give those instructions in English, just as we'd lay out a task for a child.

But human language can be ambiguous, and some instructors forget to mention important details. Suppose you told your household robot how to prepare ramen noodles, but forgot to mention heating the water or tell it where the stove is.

In his Robot Learning Lab, Ashutosh Saxena, assistant professor of computer science at Cornell University, is teaching robots to understand instructions in natural language from various speakers, account for missing information, and adapt to the environment at hand.

... more about:
»Robotics »Science »combining

Saxena and graduate students Dipendra K. Misra and Jaeyong Sung will describe their methods at the Robotics: Science and Systems conference at the University of California, Berkeley, July 12-16.

Video and abstract available at http://tellmedave.cs.cornell.edu

The robot may have a built-in programming language with commands like find (pan); grasp (pan); carry (pan, water tap); fill (pan, water); carry (pan, stove) and so on. Saxena's software translates human sentences, such as "Fill a pan with water, put it on the stove, heat the water. When it's boiling, add the noodles." into robot language. Notice that you didn't say, "Turn on the stove." The robot has to be smart enough to fill in that missing step.

Saxena's robot, equipped with a 3-D camera, scans its environment and identifies the objects in it, using computer vision software previously developed in Saxena's lab. The robot has been trained to associate objects with their capabilities: A pan can be poured into or poured from; stoves can have other objects set on them, and can heat things.

So the robot can identify the pan, locate the water faucet and stove and incorporate that information into its procedure. If you tell it to "heat water" it can use the stove or the microwave, depending on which is available. And it can carry out the same actions tomorrow if you've moved the pan, or even moved the robot to a different kitchen.

Other workers have attacked these problems by giving a robot a set of templates for common actions and chewing up sentences one word at a time. Saxena's research group uses techniques computer scientists call "machine learning" to train the robot's computer brain to associate entire commands with flexibly defined actions. The computer is fed animated video simulations of the action –- created by humans in a process similar to playing a video game – accompanied by recorded voice commands from several different speakers.

The computer stores the combination of many similar commands as a flexible pattern that can match many variations, so when it hears "Take the pot to the stove," "Carry the pot to the stove," "Put the pot on the stove," "Go to the stove and heat the pot" and so on, it calculates the probability of a match with what it has heard before, and if the probability is high enough, it declares a match. A similarly fuzzy version of the video simulation supplies a plan for the action: Wherever the sink and the stove are, the path can be matched to the recorded action of carrying the pot of water from one to the other.

Of course the robot still doesn't get it right all the time. To test, the researchers gave instructions for preparing ramen noodles and for making affogato – an Italian dessert combining coffee and ice cream: "Take some coffee in a cup. Add ice cream of your choice. Finally, add raspberry syrup to the mixture."

The robot performed correctly up to 64 percent of the time even when the commands were varied or the environment was changed, and it was able to fill in missing steps. That was three to four times better than previous methods, the researchers reported, but "There is still room for improvement."

You can teach a simulated robot to perform a kitchen task at the "Tell me Dave" website, and your input there will become part of a crowdsourced library of instructions for the Cornell robots. Aditya Jami, visiting researcher at Cornell, is helping Tell Me Dave to scale the library to millions of examples. "With crowdsourcing at such a scale, robots will learn at a much faster rate," Saxena said.

###

Cornell University has television, ISDN and dedicated Skype/Google+ Hangout studios available for media interviews.

Syl Kacapyr | Eurek Alert!

Further reports about: Robotics Science combining

More articles from Power and Electrical Engineering:

nachricht Better combustion for power generation
31.05.2016 | DOE/Oak Ridge National Laboratory

nachricht Fast, stretchy circuits could yield new wave of wearable electronics
30.05.2016 | University of Wisconsin-Madison

All articles from Power and Electrical Engineering >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Attosecond camera for nanostructures

Physicists of the Laboratory for Attosecond Physics at the Max Planck Institute of Quantum Optics and the Ludwig-Maximilians-Universität Munich in collaboration with scientists from the Friedrich-Alexander-Universität Erlangen-Nürnberg have observed a light-matter phenomenon in nano-optics, which lasts only attoseconds.

The interaction between light and matter is of key importance in nature, the most prominent example being photosynthesis. Light-matter interactions have also...

Im Focus: Worldwide Success of Tyrolean Wastewater Treatment Technology

A biological and energy-efficient process, developed and patented by the University of Innsbruck, converts nitrogen compounds in wastewater treatment facilities into harmless atmospheric nitrogen gas. This innovative technology is now being refined and marketed jointly with the United States’ DC Water and Sewer Authority (DC Water). The largest DEMON®-system in a wastewater treatment plant is currently being built in Washington, DC.

The DEMON®-system was developed and patented by the University of Innsbruck 11 years ago. Today this successful technology has been implemented in about 70...

Im Focus: Computational high-throughput screening finds hard magnets containing less rare earth elements

Permanent magnets are very important for technologies of the future like electromobility and renewable energy, and rare earth elements (REE) are necessary for their manufacture. The Fraunhofer Institute for Mechanics of Materials IWM in Freiburg, Germany, has now succeeded in identifying promising approaches and materials for new permanent magnets through use of an in-house simulation process based on high-throughput screening (HTS). The team was able to improve magnetic properties this way and at the same time replaced REE with elements that are less expensive and readily available. The results were published in the online technical journal “Scientific Reports”.

The starting point for IWM researchers Wolfgang Körner, Georg Krugel, and Christian Elsässer was a neodymium-iron-nitrogen compound based on a type of...

Im Focus: Atomic precision: technologies for the next-but-one generation of microchips

In the Beyond EUV project, the Fraunhofer Institutes for Laser Technology ILT in Aachen and for Applied Optics and Precision Engineering IOF in Jena are developing key technologies for the manufacture of a new generation of microchips using EUV radiation at a wavelength of 6.7 nm. The resulting structures are barely thicker than single atoms, and they make it possible to produce extremely integrated circuits for such items as wearables or mind-controlled prosthetic limbs.

In 1965 Gordon Moore formulated the law that came to be named after him, which states that the complexity of integrated circuits doubles every one to two...

Im Focus: Researchers demonstrate size quantization of Dirac fermions in graphene

Characterization of high-quality material reveals important details relevant to next generation nanoelectronic devices

Quantum mechanics is the field of physics governing the behavior of things on atomic scales, where things work very differently from our everyday world.

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

Networking 4.0: International Laser Technology Congress AKL’16 Shows New Ways of Cooperations

24.05.2016 | Event News

Challenges of rural labor markets

20.05.2016 | Event News

International expert meeting “Health Business Connect” in France

19.05.2016 | Event News

 
Latest News

Better combustion for power generation

31.05.2016 | Power and Electrical Engineering

Stick insects produce bacterial enzymes themselves

31.05.2016 | Life Sciences

In a New Method for Searching Image Databases, a Hand-drawn Sketch Is all it Takes

31.05.2016 | Information Technology

VideoLinks
B2B-VideoLinks
More VideoLinks >>>