New world record in energy-efficient data processing

Scientists from Frankfurt´s Goethe University and the Karlsruhe Institute of Technology (KIT) developed a system that substantially reduces the energy consumption for processing huge amounts of data.

They improved over the power efficiency of the former record holders from Stanford University by a factor of three to four. The record is listed in the “sort benchmark”, which is published by companies like Hewlett-Packard und Microsoft.

The team around Prof. Ulrich Meyer from Goethe University and Prof. Peter Sanders from KIT enabled the record by using seemingly unconventional hardware: instead of server processors with high power requirements, the computer scientists took processors of type Intel Atom. These are microprocessors originally developed for netbooks.

Their lower processing power compared to server systems was compensated by the usage of highly efficient algorithms. Instead of hard drives, which consume a lot of power for the mechanics, the team employed so-called Solid State Disks (SSD), which are clearly faster and, at the same time, more power-economical.

The record pops the question if the increasing hunger for energy in information technology could be strongly reduced. “In the long run, many small, power-efficient and cooperating systems are going to replace the so far used, heavy weighted ones”, explains Peter Sanders.

Starting point for their research project was one of the key problems in computer science, namely sorting of data. Computers connected via Internet generate constantly growing amounts of data. In order to enable analysis of the data, it has to be sorted according to a specific criterion first. The efficient sorting of data is thus of central interest for search engines and databases – and therefore an important research topic in both theoretical and practical computer science.

In the three categories of the competition, the researchers had to sort data amounts of 10GB, 100GB and 1TB, respectively, consisting of datasets with 100 Byte each. Even in the largest category of 1 Terabyte, which corresponds to a stack of paper of 10km height, the new record holders only spent 0,2 kWh. This is about the energy needed to boil 2 liters of water.

Supervised by Sanders and Meyer, the Ph.D. candidates Johannes Singler (KIT) and Andreas Beckmann (Goethe University) developed the energy-saving system. The research groups of both universities are internationally noted for their work on the design and implementation of efficient algorithms for processing large data.

The world records are listed as “JouleSort” entries in the »Sort Benchmark. For further information please see http://sortbenchmark.org