Machine learning at speed
Inserting lightweight optimization code in high-speed network devices has enabled a KAUST-led collaboration to increase the speed of machine learning on parallelized computing systems five-fold.
This “in-network aggregation” technology, developed with researchers and systems architects at Intel, Microsoft and the University of Washington, can provide dramatic speed improvements using readily available programmable network hardware.
The fundamental benefit of artificial intelligence (AI) that gives it so much power to “understand” and interact with the world is the machine-learning step, in which the model is trained using large sets of labeled training data. The more data the AI is trained on, the better the model is likely to perform when exposed to new inputs.
The recent burst of AI applications is largely due to better machine learning and the use of larger models and more diverse datasets. Performing the machine-learning computations, however, is an enormously taxing task that increasingly relies on large arrays of computers running the learning algorithm in parallel.
“How to train deep-learning models at a large scale is a very challenging problem,” says Marco Canini from the KAUST research team. “The AI models can consist of billions of parameters, and we can use hundreds of processors that need to work efficiently in parallel. In such systems, communication among processors during incremental model updates easily becomes a major performance bottleneck.”
The team found a potential solution in new network technology developed by Barefoot Networks, a division of Intel.
“We use Barefoot Networks’ new programmable dataplane networking hardware to offload part of the work performed during distributed machine-learning training,” explains Amedeo Sapio, a KAUST alumnus who has since joined the Barefoot Networks team at Intel. “Using this new programmable networking hardware, rather than just the network, to move data means that we can perform computations along the network paths.”
The key innovation of the team’s SwitchML platform is to allow the network hardware to perform the data aggregation task at each synchronization step during the model update phase of the machine-learning process. Not only does this offload part of the computational load, it also significantly reduces the amount of data transmission.
“Although the programmable switch dataplane can do operations very quickly, the operations it can do are limited,” says Canini. “So our solution had to be simple enough for the hardware and yet flexible enough to solve challenges such as limited onboard memory capacity. SwitchML addresses this challenge by co-designing the communication network and the distributed training algorithm, achieving an acceleration of up to 5.5 times compared to the state-of-the-art approach.”
All latest news from the category: Information Technology
Here you can find a summary of innovations in the fields of information and data processing and up-to-date developments on IT equipment and hardware.
This area covers topics such as IT services, IT architectures, IT management and telecommunications.
Newest articles
Hurricane Helene’s gravity waves revealed by NASA’s AWE
On Sept. 26, 2024, as Hurricane Helene slammed into the Gulf Coast of Florida, NASA’s Atmospheric Waves Experiment, or AWE, recorded enormous swells in the atmosphere that the hurricane produced…
Researchers develop robotic sensory cilia
… that monitor internal biomarkers to detect and assess airway diseases. Xiaoguang Dong, assistant professor of mechanical engineering, is leading a team of researchers that has developed a system of…
Plastic device aids robot-assisted heart surgery
Surgical field expansion plate allows surgeons more independence. Robot-assisted heart surgery usually requires an assistant at the operating table to help the surgeon insert the robot arm through a small…