A group of scientists from Russia and Greece opens up new prospects for using local learning rules based on memristors to solve artificial intelligence problems using complex spiking neural network architectures
Lobachevsky University scientists together with their colleagues from the National Research Center "Kurchatov Institute" (Moscow) and the National Research Center "Demokritos" (Athens) are working on the hardware implementation of a spiking neural network based on memristors.
Cross-section image of the metal-oxide-metal memristive structure based on ZrO2(Y) polycrystalline film (a); corresponding schematic view of the cross-point memristive device (b); STDP dependencies of memristive device conductance changes for different delay values between pre- and postsynaptic neuron spikes (c); photographs of a microchip and an array of memristive devices in a standard cermet casing (d); the simplest spiking neural network architecture learning on the basis of local rules for changing memristive weights (e).
Credit: Lobachevsky University
The key elements of such a network, along with pulsed neurons, are artificial synaptic connections that can change the strength (weight) of connection between neurons during the learning.
For this purpose, memristive devices based on metal-oxide-metal nanostructures developed at the UNN Physics and Technology Research Institute (PTRI) are suitable, but their use in specific spiking neural network architectures developed at the Kurchatov Institute requires demonstration of biologically plausible learning principles.
The biological mechanism of learning of neural systems is described by Hebb's rule, according to which learning occurs as a result of an increase in the strength of connection (synaptic weight) between simultaneously active neurons, which indicates the presence of a causal relationship in their excitation.
One of the clarifying forms of this fundamental rule is plasticity, which depends on the time of arrival of pulses (Spike-Timing Dependent Plasticity - STDP).
In accordance with STDP, synaptic weight increases if the postsynaptic neuron generates a pulse (spike) immediately after the presynaptic one, and vice versa, the synaptic weight decreases if the postsynaptic neuron generates a spike right before the presynaptic one. Moreover, the smaller the time difference Δt between the pre- and postsynaptic spikes, the more pronounced the weight change will be.
According to one of the researchers, Head of the UNN PTRI laboratory Alexei Mikhailov, in order to demonstrate the STDP principle, memristive nanostructures based on yttria-stabilized zirconia (YSZ) thin films were used. YSZ is a well-known solid-state electrolyte with high oxygen ion mobility.
"Due to a specified concentration of oxygen vacancies, which is determined by the controlled concentration of yttrium impurities, and the heterogeneous structure of the films obtained by magnetron sputtering, such memristive structures demonstrate controlled bipolar switching between different resistive states in a wide resistance range. The switching is associated with the formation and destruction of conductive channels along grain boundaries in the polycrystalline ZrO2 (Y) film," notes Alexei Mikhailov.
An array of memristive devices for research was implemented in the form of a microchip mounted in a standard cermet casing, which facilitates the integration of the array into a neural network's analog circuit. The full technological cycle for creating memristive microchips is currently implemented at the UNN PTRI. In the future, it is possible to scale the devices down to the minimum size of about 50 nm, as was established by Greek partners.
Our studies of the dynamic plasticity of the memoristive devices, continues Alexey Mikhailov, have shown that the form of the conductance change depending on Δt is in good agreement with the STDP learning rules. It should be also noted that if the initial value of the memristor conductance is close to the maximum, it is easy to reduce the corresponding weight while it is difficult to enhance it, and in the case of a memristor with a minimum conductance in the initial state, it is difficult to reduce its weight, but it is easy to enhance it.
According to Vyacheslav Demin, director-coordinator in the area of nature-like technologies of the Kurchatov Institute, who is one of the ideologues of this work, the established pattern of change in the memristor conductance clearly demonstrates the possibility of hardware implementation of the so-called local learning rules. Such rules for changing the strength of synaptic connections depend only on the values of variables that are present locally at each time point (neuron activities and current weights).
"This essentially distinguishes such principle from the traditional learning algorithm, which is based on global rules for changing weights, using information on the error values at the current time point for each neuron of the output neural network layer (in a widely popular group of error back propagation methods). The traditional principle is not biosimilar, it requires "external" (expert) knowledge of the correct answers for each example presented to the network (that is, they do not have the property of self-learning). This principle is difficult to implement on the basis of memristors, since it requires controlled precise changes of memristor conductances, as opposed to local rules. Such precise control is not always possible due to the natural variability (a wide range of parameters) of memristors as analog elements," says Vyacheslav Demin.
Local learning rules of the STDP type implemented in hardware on memristors provide the basis for autonomous ("unsupervised") learning of a spiking neural network. In this case, the final state of the network does not depend on its initial state, but depends only on the learning conditions (a specific sequence of pulses). According to Vyacheslav Demin, this opens up prospects for the application of local learning rules based on memristors when solving artificial intelligence problems with the use of complex spiking neural network architectures.
A paper on the results of this research was recently published in the journal Microelectronic Engineering.
Nikita Avralev | EurekAlert!
Numbers count in the genetics of moles and melanomas
16.08.2019 | University of Queensland
Working out why plants get sick
16.08.2019 | Institut für Pflanzenbiochemie
Soft robots have a distinct advantage over their rigid forebears: they can adapt to complex environments, handle fragile objects and interact safely with humans. Made from silicone, rubber or other stretchable polymers, they are ideal for use in rehabilitation exoskeletons and robotic clothing. Soft bio-inspired robots could one day be deployed to explore remote or dangerous environments.
Most soft robots are actuated by rigid, noisy pumps that push fluids into the machines' moving parts. Because they are connected to these bulky pumps by tubes,...
Researchers at TU Graz are working together with European partners on new possibilities of measuring vehicle emissions.
Today, air pollution is one of the biggest challenges facing European cities. As part of the Horizon 2020 research project CARES (City Air Remote Emission...
Over the next three years, researchers from the Vrije Universiteit Brussel, University of Cambridge, École Supérieure de Physique et de Chimie Industrielles de la ville de Paris (ESPCI-Paris) and Empa will be working together with the Dutch Polymer manufacturer SupraPolix on the next generation of robots: (soft) robots that ‘feel pain’ and heal themselves. The partners can count on 3 million Euro in support from the European Commission.
Soon robots will not only be found in factories and laboratories, but will be assisting us in our immediate environment. They will help us in the household, to...
Scientists at the University of Leeds have created a new form of gold which is just two atoms thick - the thinnest unsupported gold ever created.
The researchers measured the thickness of the gold to be 0.47 nanometres - that is one million times thinner than a human finger nail. The material is regarded...
An international team of scientists involving the Max Planck Institute for the Structure and Dynamics of Matter (MPSD) in Hamburg has unraveled the light-induced electron-localization dynamics in transition metals at the attosecond timescale. The team investigated for the first time the many-body electron dynamics in transition metals before thermalization sets in. Their work has now appeared in Nature Physics.
The researchers from ETH Zurich (Switzerland), the MPSD (Germany), the Center for Computational Sciences of University of Tsukuba (Japan) and the Center for...
16.08.2019 | Event News
14.08.2019 | Event News
12.08.2019 | Event News
16.08.2019 | Life Sciences
16.08.2019 | Physics and Astronomy
16.08.2019 | Medical Engineering