Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Researchers incorporate computer vision and uncertainty into AI for robotic prosthetics

28.05.2020

Researchers have developed new software that can be integrated with existing hardware to enable people using robotic prosthetics or exoskeletons to walk in a safer, more natural manner on different types of terrain. The new framework incorporates computer vision into prosthetic leg control, and includes robust artificial intelligence (AI) algorithms that allow the software to better account for uncertainty.

"Lower-limb robotic prosthetics need to execute different behaviors based on the terrain users are walking on," says Edgar Lobaton, co-author of a paper on the work and an associate professor of electrical and computer engineering at North Carolina State University.


Imaging devices and environmental context. (a) On-glasses camera configuration using a Tobii Pro Glasses 2 eye tracker. (b) Lower limb data acquisition device with a camera and an IMU chip. (c) and (d) Example frames from the cameras for the two data acquisition configurations. (e) and (f) Example images of the data collection environment and terrains considered in the experiments.

Credit: Edgar Lobaton

"The framework we've created allows the AI in robotic prostheses to predict the type of terrain users will be stepping on, quantify the uncertainties associated with that prediction, and then incorporate that uncertainty into its decision-making."

The researchers focused on distinguishing between six different terrains that require adjustments in a robotic prosthetic's behavior: tile, brick, concrete, grass, "upstairs" and "downstairs."

"If the degree of uncertainty is too high, the AI isn't forced to make a questionable decision - it could instead notify the user that it doesn't have enough confidence in its prediction to act, or it could default to a 'safe' mode," says Boxuan Zhong, lead author of the paper and a recent Ph.D. graduate from NC State.

The new "environmental context" framework incorporates both hardware and software elements. The researchers designed the framework for use with any lower-limb robotic exoskeleton or robotic prosthetic device, but with one additional piece of hardware: a camera.

In their study, the researchers used cameras worn on eyeglasses and cameras mounted on the lower-limb prosthesis itself. The researchers evaluated how the AI was able to make use of computer vision data from both types of camera, separately and when used together.

"Incorporating computer vision into control software for wearable robotics is an exciting new area of research," says Helen Huang, a co-author of the paper. "We found that using both cameras worked well, but required a great deal of computing power and may be cost prohibitive.

However, we also found that using only the camera mounted on the lower limb worked pretty well - particularly for near-term predictions, such as what the terrain would be like for the next step or two."

Huang is the Jackson Family Distinguished Professor of Biomedical Engineering in the Joint Department of Biomedical Engineering at NC State and the University of North Carolina at Chapel Hill.

The most significant advance, however, is to the AI itself.

"We came up with a better way to teach deep-learning systems how to evaluate and quantify uncertainty in a way that allows the system to incorporate uncertainty into its decision making," Lobaton says. "This is certainly relevant for robotic prosthetics, but our work here could be applied to any type of deep-learning system."

To train the AI system, researchers connected the cameras to able-bodied individuals, who then walked through a variety of indoor and outdoor environments. The researchers then did a proof-of-concept evaluation by having a person with lower-limb amputation wear the cameras while traversing the same environments.

"We found that the model can be appropriately transferred so the system can operate with subjects from different populations," Lobaton says. "That means that the AI worked well even thought it was trained by one group of people and used by somebody different."

However, the new framework has not yet been tested in a robotic device.

"We are excited to incorporate the framework into the control system for working robotic prosthetics - that's the next step," Huang says.

"And we're also planning to work on ways to make the system more efficient, in terms of requiring less visual data input and less data processing," says Zhong.

###

The paper, "Environmental Context Prediction for Lower Limb Prostheses with Uncertainty Quantification," is published in IEEE Transactions on Automation Science and Engineering. The paper was co-authored by Rafael da Silva, a Ph.D. student at NC State; and Minhan Li, a Ph.D. student in the Joint Department of Biomedical Engineering.

The work was done with support from the National Science Foundation under grants 1552828, 1563454 and 1926998.

Matt Shipman | EurekAlert!
Further information:
https://news.ncsu.edu/2020/05/prosthetics-computer-vision-uncertainty/
http://dx.doi.org/10.1109/TASE.2020.2993399

More articles from Information Technology:

nachricht Color barcode becomes ISO standard
14.07.2020 | Fraunhofer-Institut für Sichere Informationstechnologie SIT

nachricht Virtual reality can assist with the evaluation of future transport concepts
13.07.2020 | Fraunhofer-Institut für Arbeitswirtschaft und Organisation IAO

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Electron cryo-microscopy: Using inexpensive technology to produce high-resolution images

Biochemists at Martin Luther University Halle-Wittenberg (MLU) have used a standard electron cryo-microscope to achieve surprisingly good images that are on par with those taken by far more sophisticated equipment. They have succeeded in determining the structure of ferritin almost at the atomic level. Their results were published in the journal "PLOS ONE".

Electron cryo-microscopy has become increasingly important in recent years, especially in shedding light on protein structures. The developers of the new...

Im Focus: The spin state story: Observation of the quantum spin liquid state in novel material

New insight into the spin behavior in an exotic state of matter puts us closer to next-generation spintronic devices

Aside from the deep understanding of the natural world that quantum physics theory offers, scientists worldwide are working tirelessly to bring forth a...

Im Focus: Excitation of robust materials

Kiel physics team observed extremely fast electronic changes in real time in a special material class

In physics, they are currently the subject of intensive research; in electronics, they could enable completely new functions. So-called topological materials...

Im Focus: Electrons in the fast lane

Solar cells based on perovskite compounds could soon make electricity generation from sunlight even more efficient and cheaper. The laboratory efficiency of these perovskite solar cells already exceeds that of the well-known silicon solar cells. An international team led by Stefan Weber from the Max Planck Institute for Polymer Research (MPI-P) in Mainz has found microscopic structures in perovskite crystals that can guide the charge transport in the solar cell. Clever alignment of these "electron highways" could make perovskite solar cells even more powerful.

Solar cells convert sunlight into electricity. During this process, the electrons of the material inside the cell absorb the energy of the light....

Im Focus: The lightest electromagnetic shielding material in the world

Empa researchers have succeeded in applying aerogels to microelectronics: Aerogels based on cellulose nanofibers can effectively shield electromagnetic radiation over a wide frequency range – and they are unrivalled in terms of weight.

Electric motors and electronic devices generate electromagnetic fields that sometimes have to be shielded in order not to affect neighboring electronic...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Contact Tracing Apps against COVID-19: German National Academy Leopoldina hosts international virtual panel discussion

07.07.2020 | Event News

International conference QuApps shows status quo of quantum technology

02.07.2020 | Event News

Dresden Nexus Conference 2020: Same Time, Virtual Format, Registration Opened

19.05.2020 | Event News

 
Latest News

Shedding light on the brown color of algae

14.07.2020 | Life Sciences

Color barcode becomes ISO standard

14.07.2020 | Information Technology

New substance library to accelerate the search for active compounds

14.07.2020 | Life Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>