A group of scientists from Russia and Greece opens up new prospects for using local learning rules based on memristors to solve artificial intelligence problems using complex spiking neural network architectures
Lobachevsky University scientists together with their colleagues from the National Research Center "Kurchatov Institute" (Moscow) and the National Research Center "Demokritos" (Athens) are working on the hardware implementation of a spiking neural network based on memristors.
Cross-section image of the metal-oxide-metal memristive structure based on ZrO2(Y) polycrystalline film (a); corresponding schematic view of the cross-point memristive device (b); STDP dependencies of memristive device conductance changes for different delay values between pre- and postsynaptic neuron spikes (c); photographs of a microchip and an array of memristive devices in a standard cermet casing (d); the simplest spiking neural network architecture learning on the basis of local rules for changing memristive weights (e).
Credit: Lobachevsky University
The key elements of such a network, along with pulsed neurons, are artificial synaptic connections that can change the strength (weight) of connection between neurons during the learning.
For this purpose, memristive devices based on metal-oxide-metal nanostructures developed at the UNN Physics and Technology Research Institute (PTRI) are suitable, but their use in specific spiking neural network architectures developed at the Kurchatov Institute requires demonstration of biologically plausible learning principles.
The biological mechanism of learning of neural systems is described by Hebb's rule, according to which learning occurs as a result of an increase in the strength of connection (synaptic weight) between simultaneously active neurons, which indicates the presence of a causal relationship in their excitation.
One of the clarifying forms of this fundamental rule is plasticity, which depends on the time of arrival of pulses (Spike-Timing Dependent Plasticity - STDP).
In accordance with STDP, synaptic weight increases if the postsynaptic neuron generates a pulse (spike) immediately after the presynaptic one, and vice versa, the synaptic weight decreases if the postsynaptic neuron generates a spike right before the presynaptic one. Moreover, the smaller the time difference Δt between the pre- and postsynaptic spikes, the more pronounced the weight change will be.
According to one of the researchers, Head of the UNN PTRI laboratory Alexei Mikhailov, in order to demonstrate the STDP principle, memristive nanostructures based on yttria-stabilized zirconia (YSZ) thin films were used. YSZ is a well-known solid-state electrolyte with high oxygen ion mobility.
"Due to a specified concentration of oxygen vacancies, which is determined by the controlled concentration of yttrium impurities, and the heterogeneous structure of the films obtained by magnetron sputtering, such memristive structures demonstrate controlled bipolar switching between different resistive states in a wide resistance range. The switching is associated with the formation and destruction of conductive channels along grain boundaries in the polycrystalline ZrO2 (Y) film," notes Alexei Mikhailov.
An array of memristive devices for research was implemented in the form of a microchip mounted in a standard cermet casing, which facilitates the integration of the array into a neural network's analog circuit. The full technological cycle for creating memristive microchips is currently implemented at the UNN PTRI. In the future, it is possible to scale the devices down to the minimum size of about 50 nm, as was established by Greek partners.
Our studies of the dynamic plasticity of the memoristive devices, continues Alexey Mikhailov, have shown that the form of the conductance change depending on Δt is in good agreement with the STDP learning rules. It should be also noted that if the initial value of the memristor conductance is close to the maximum, it is easy to reduce the corresponding weight while it is difficult to enhance it, and in the case of a memristor with a minimum conductance in the initial state, it is difficult to reduce its weight, but it is easy to enhance it.
According to Vyacheslav Demin, director-coordinator in the area of nature-like technologies of the Kurchatov Institute, who is one of the ideologues of this work, the established pattern of change in the memristor conductance clearly demonstrates the possibility of hardware implementation of the so-called local learning rules. Such rules for changing the strength of synaptic connections depend only on the values of variables that are present locally at each time point (neuron activities and current weights).
"This essentially distinguishes such principle from the traditional learning algorithm, which is based on global rules for changing weights, using information on the error values at the current time point for each neuron of the output neural network layer (in a widely popular group of error back propagation methods). The traditional principle is not biosimilar, it requires "external" (expert) knowledge of the correct answers for each example presented to the network (that is, they do not have the property of self-learning). This principle is difficult to implement on the basis of memristors, since it requires controlled precise changes of memristor conductances, as opposed to local rules. Such precise control is not always possible due to the natural variability (a wide range of parameters) of memristors as analog elements," says Vyacheslav Demin.
Local learning rules of the STDP type implemented in hardware on memristors provide the basis for autonomous ("unsupervised") learning of a spiking neural network. In this case, the final state of the network does not depend on its initial state, but depends only on the learning conditions (a specific sequence of pulses). According to Vyacheslav Demin, this opens up prospects for the application of local learning rules based on memristors when solving artificial intelligence problems with the use of complex spiking neural network architectures.
A paper on the results of this research was recently published in the journal Microelectronic Engineering.
Nikita Avralev | EurekAlert!
New yeast species discovered in Braunschweig, Germany
13.12.2019 | Leibniz-Institut DSMZ-Deutsche Sammlung von Mikroorganismen und Zellkulturen GmbH
Saliva test shows promise for earlier and easier detection of mouth and throat cancer
13.12.2019 | Elsevier
Vaccinia viruses serve as a vaccine against human smallpox and as the basis of new cancer therapies. Two studies now provide fascinating insights into their unusual propagation strategy at the atomic level.
For viruses to multiply, they usually need the support of the cells they infect. In many cases, only in their host’s nucleus can they find the machines,...
More than one hundred and fifty years have passed since the publication of James Clerk Maxwell's "A Dynamical Theory of the Electromagnetic Field" (1865). What would our lives be without this publication?
It is difficult to imagine, as this treatise revolutionized our fundamental understanding of electric fields, magnetic fields, and light. The twenty original...
In a joint experimental and theoretical work performed at the Heidelberg Max Planck Institute for Nuclear Physics, an international team of physicists detected for the first time an orbital crossing in the highly charged ion Pr⁹⁺. Optical spectra were recorded employing an electron beam ion trap and analysed with the aid of atomic structure calculations. A proposed nHz-wide transition has been identified and its energy was determined with high precision. Theory predicts a very high sensitivity to new physics and extremely low susceptibility to external perturbations for this “clock line” making it a unique candidate for proposed precision studies.
Laser spectroscopy of neutral atoms and singly charged ions has reached astonishing precision by merit of a chain of technological advances during the past...
The ability to investigate the dynamics of single particle at the nano-scale and femtosecond level remained an unfathomed dream for years. It was not until the dawn of the 21st century that nanotechnology and femtoscience gradually merged together and the first ultrafast microscopy of individual quantum dots (QDs) and molecules was accomplished.
Ultrafast microscopy studies entirely rely on detecting nanoparticles or single molecules with luminescence techniques, which require efficient emitters to...
Graphene, a two-dimensional structure made of carbon, is a material with excellent mechanical, electronic and optical properties. However, it did not seem suitable for magnetic applications. Together with international partners, Empa researchers have now succeeded in synthesizing a unique nanographene predicted in the 1970s, which conclusively demonstrates that carbon in very specific forms has magnetic properties that could permit future spintronic applications. The results have just been published in the renowned journal Nature Nanotechnology.
Depending on the shape and orientation of their edges, graphene nanostructures (also known as nanographenes) can have very different properties – for example,...
03.12.2019 | Event News
15.11.2019 | Event News
15.11.2019 | Event News
13.12.2019 | Physics and Astronomy
13.12.2019 | Physics and Astronomy
13.12.2019 | Materials Sciences