Optical training of neural networks could lead to more efficient artificial intelligence
Researchers have shown that it is possible to train artificial neural networks directly on an optical chip. The significant breakthrough demonstrates that an optical circuit can perform a critical function of an electronics-based artificial neural network and could lead to less expensive, faster and more energy efficient ways to perform complex tasks such as speech or image recognition.
Researchers have shown a neural network can be trained using an optical circuit (blue rectangle in the illustration). In the full network there would be several of these linked together. The laser inputs (green) encode information that is carried through the chip by optical waveguides (black). The chip performs operations crucial to the artificial neural network using tunable beam splitters, which are represented by the curved sections in the waveguides. These sections couple two adjacent waveguides together and are tuned by adjusting the settings of optical phase shifters (red and blue glowing objects), which act like 'knobs' that can be adjusted during training to perform a given task.
Credit: Tyler W. Hughes, Stanford University
"Using an optical chip to perform neural network computations more efficiently than is possible with digital computers could allow more complex problems to be solved," said research team leader Shanhui Fan of Stanford University. "This would enhance the capability of artificial neural networks to perform tasks required for self-driving cars or to formulate an appropriate response to a spoken question, for example. It could also improve our lives in ways we can't imagine now."
An artificial neural network is a type of artificial intelligence that uses connected units to process information in a manner similar to the way the brain processes information. Using these networks to perform a complex task, for instance voice recognition, requires the critical step of training the algorithms to categorize inputs, such as different words.
Although optical artificial neural networks were recently demonstrated experimentally, the training step was performed using a model on a traditional digital computer and the final settings were then imported into the optical circuit. In Optica, The Optical Society's journal for high impact research, Stanford University researchers report a method for training these networks directly in the device by implementing an optical analogue of the 'backpropagation' algorithm, which is the standard way to train conventional neural networks.
"Using a physical device rather than a computer model for training makes the process more accurate," said Tyler W. Hughes, first author of the paper. "Also, because the training step is a very computationally expensive part of the implementation of the neural network, performing this step optically is key to improving the computational efficiency, speed and power consumption of artificial networks."
A light-based network
Although neural network processing is typically performed using a traditional computer, there are significant efforts to design hardware optimized specifically for neural network computing. Optics-based devices are of great interest because they can perform computations in parallel while using less energy than electronic devices.
In the new work, the researchers overcame a significant challenge to implementing an all-optical neural network by designing an optical chip that replicates the way that conventional computers train neural networks.
An artificial neural network can be thought of as a black box with a number of knobs. During the training step, these knobs are each turned a little and then the system is tested to see if the performance of the algorithms improved.
"Our method not only helps predict which direction to turn the knobs but also how much you should turn each knob to get you closer to the desired performance," said Hughes. "Our approach speeds up training significantly, especially for large networks, because we get information about each knob in parallel."
The new training protocol operates on optical circuits with tunable beam splitters that are adjusted by changing the settings of optical phase shifters. Laser beams encoding information to be processed are fired into the optical circuit and carried by optical waveguides through the beam splitters, which are adjusted like knobs to train the neural network algorithms.
In the new training protocol, the laser is first fed through the optical circuit. Upon exiting the device, the difference from the expected outcome is calculated. This information is then used to generate a new light signal, which is sent back through the optical network in the opposite direction. By measuring the optical intensity around each beam splitter during this process, the researchers showed how to detect, in parallel, how the neural network performance will change with respect to each beam splitter's setting. The phase shifter settings can be changed based on this information, and the process may be repeated until the neural network produces the desired outcome.
The researchers tested their training technique with optical simulations by teaching an algorithm to perform complicated functions, such as picking out complex features within a set of points. They found that the optical implementation performed similarly to a conventional computer.
"Our work demonstrates that you can use the laws of physics to implement computer science algorithms," said Fan. "By training these networks in the optical domain, it shows that optical neural network systems could be built to carry out certain functionalities using optics alone."
The researchers plan to further optimize the system and want to use it to implement a practical application of a neural network task. The general approach they designed could be used with various neural network architectures and for other applications such as reconfigurable optics.
Paper: T. W. Hughes, M. Minkov, Y. Shi, S. Fan, "Training of photonic neural networks through in situ backpropagation and gradient measurement," Optica, Volume 5,Issue , pages 864-871 (2018)
Optica is an open-access, online-only journal dedicated to the rapid dissemination of high-impact peer-reviewed research across the entire spectrum of optics and photonics. Published monthly by The Optical Society (OSA), Optica provides a forum for pioneering research to be swiftly accessed by the international community, whether that research is theoretical or experimental, fundamental or applied. Optica maintains a distinguished editorial board of more than 50 associate editors from around the world and is overseen by Editor-in-Chief Alex Gaeta, Columbia University, USA. For more information, visit Optica.
About The Optical Society
Founded in 1916, The Optical Society (OSA) is the leading professional organization for scientists, engineers, students and business leaders who fuel discoveries, shape real-life applications and accelerate achievements in the science of light. Through world-renowned publications, meetings and membership initiatives, OSA provides quality research, inspired interactions and dedicated resources for its extensive global network of optics and photonics experts. For more information, visit osa.org.
Azalea Coste | EurekAlert!
'Building up' stretchable electronics to be as multipurpose as your smartphone
14.08.2018 | University of California - San Diego
New interactive machine learning tool makes car designs more aerodynamic
14.08.2018 | Institute of Science and Technology Austria
Scientists develop first tool to use machine learning methods to compute flow around interactively designable 3D objects. Tool will be presented at this year’s prestigious SIGGRAPH conference.
When engineers or designers want to test the aerodynamic properties of the newly designed shape of a car, airplane, or other object, they would normally model...
Researchers from TU Graz and their industry partners have unveiled a world first: the prototype of a robot-controlled, high-speed combined charging system (CCS) for electric vehicles that enables series charging of cars in various parking positions.
Global demand for electric vehicles is forecast to rise sharply: by 2025, the number of new vehicle registrations is expected to reach 25 million per year....
Proteins must be folded correctly to fulfill their molecular functions in cells. Molecular assistants called chaperones help proteins exploit their inbuilt folding potential and reach the correct three-dimensional structure. Researchers at the Max Planck Institute of Biochemistry (MPIB) have demonstrated that actin, the most abundant protein in higher developed cells, does not have the inbuilt potential to fold and instead requires special assistance to fold into its active state. The chaperone TRiC uses a previously undescribed mechanism to perform actin folding. The study was recently published in the journal Cell.
Actin is the most abundant protein in highly developed cells and has diverse functions in processes like cell stabilization, cell division and muscle...
Scientists have discovered that the electrical resistance of a copper-oxide compound depends on the magnetic field in a very unusual way -- a finding that could help direct the search for materials that can perfectly conduct electricity at room temperatur
What happens when really powerful magnets--capable of producing magnetic fields nearly two million times stronger than Earth's--are applied to materials that...
The quality of materials often depends on the manufacturing process. In casting and welding, for example, the rate at which melts solidify and the resulting microstructure of the alloy is important. With metallic foams as well, it depends on exactly how the foaming process takes place. To understand these processes fully requires fast sensing capability. The fastest 3D tomographic images to date have now been achieved at the BESSY II X-ray source operated by the Helmholtz-Zentrum Berlin.
Dr. Francisco Garcia-Moreno and his team have designed a turntable that rotates ultra-stably about its axis at a constant rotational speed. This really depends...
08.08.2018 | Event News
27.07.2018 | Event News
25.07.2018 | Event News
15.08.2018 | Physics and Astronomy
15.08.2018 | Materials Sciences
15.08.2018 | Life Sciences