Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

The world’s largest archive with up to 500 Petabyte of climate data is located in Hamburg

08.05.2015

DKRZ, supported by IBM, took in its new Hierarchical Storage Management System based on a High Performance Storage Systems (HPSS). Experts from IBM/Global Technology Services have implemented the latest HPSS software and the Linux operation system by Red Hat on the IBM x86 server as well as a HPSS disk cache with five Petabyte.

The new system allows currently a transfer of climate data between the high performance computer and the archive system with a bidirectional bandwidth up to 12 Gigabyte per second, increasing up to 18 Gigabyte per second in autumn 2015.


One of the disc enclosures of the 5 petabytes HPSS disk cache at DKRZ

Böttinger, DKRZ


HPSS Hardware installed at DKRZ: servers (front left) and additional racks with a total of 5 petabytes disk cache

Böttinger, DKRZ

The system manages all archived simulation data at DKRZ and serves as the central in- and output interface of the tape library.

Hamburg, May, 8th, 2015: The German Climate Computing Center (DKRZ), supported by IBM, took in operation the first configuration level of its new Hierarchical Storage Management System (HSM) that is based on a High Performance Storage Systems (HPSS). The system manages all archived simulation data at DKRZ and serves as the central in- and output interface of the tape library.

Already in autumn 2014, experts from IBM/Global Technology Services have implemented the latest HPSS software and the Linux operation system by Red Hat on the IBM x86 server as well as a HPSS disk cache with five Petabyte. The size of the disk cache, which serves as a temporary buffer, corresponds to the storage capacity of 5,000 well equipped notebook computers.

IBM specialists have migrated the data of the former, significantly smaller HPSS system to the new system within only two days of operational interruption. The new system with an essentially higher sustained throughput (in- and output) allows a transfer of climate data between the high performance computer and the archive system with a bidirectional bandwidth up to 12 Gigabyte per second. Until autumn 2015 the system will be once more upgraded to a bandwidth up to 18 Gigabyte per second (peak).

At the moment the tape library at DKRZ contains about 20 million data with a data volume of 40 Petabyte. Using HPSS DKRZ will be able to handle the massive data growth in the climate research. Within the next five years a data production of annually 75 Petabyte and a total volume of 500 Petabyte is expected, appropriate to the estimated growth of data production on the new high performance computer system for earth system research 3 (HLRE-3) being currently installed at DKRZ.

The upgrade of the HSM system is part of the project HLRE-3 that is funded with 41 million euro – two thirds by the Federal Ministry of Education and Research and one third by the Helmholtz Association. The financial resources are passed on to DKRZ by the Helmholtz Center Geesthacht – Center for Materials and Coastal Research.

HPSS results from a 20 years lasting cooperation between IBM and the five national US-Laboratories of the Department of Energy (DOE): Lawrence Livermore, Los Alamos, Sandia, Lawrence Berkley and Oak Ridge (further information: www.hpss-collaboration.org ).

HPSS is implemented for very large scientific data archives at research facilities and universities, for weather forecast systems as well as in the field of defense and national security. Organizations with CPU- and data intensive processing use HPSS. In Europe and Germany it is used among other by the European Centre for Medium Range Weather Forecasts (ECMWF), the Karlsruhe Institute for Technology (KIT), the German Meteorological Service (DWD) and the computing center Rechenzentrum Garching of the Max Planck Society (RZG).

Weitere Informationen:

https://www.dkrz.de/Klimarechner-en/datenarchiv

Jana Meyer | idw - Informationsdienst Wissenschaft

More articles from Information Technology:

nachricht NASA CubeSat to test miniaturized weather satellite technology
10.11.2017 | NASA/Goddard Space Flight Center

nachricht New approach uses light instead of robots to assemble electronic components
08.11.2017 | The Optical Society

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: A “cosmic snake” reveals the structure of remote galaxies

The formation of stars in distant galaxies is still largely unexplored. For the first time, astron-omers at the University of Geneva have now been able to closely observe a star system six billion light-years away. In doing so, they are confirming earlier simulations made by the University of Zurich. One special effect is made possible by the multiple reflections of images that run through the cosmos like a snake.

Today, astronomers have a pretty accurate idea of how stars were formed in the recent cosmic past. But do these laws also apply to older galaxies? For around a...

Im Focus: Visual intelligence is not the same as IQ

Just because someone is smart and well-motivated doesn't mean he or she can learn the visual skills needed to excel at tasks like matching fingerprints, interpreting medical X-rays, keeping track of aircraft on radar displays or forensic face matching.

That is the implication of a new study which shows for the first time that there is a broad range of differences in people's visual ability and that these...

Im Focus: Novel Nano-CT device creates high-resolution 3D-X-rays of tiny velvet worm legs

Computer Tomography (CT) is a standard procedure in hospitals, but so far, the technology has not been suitable for imaging extremely small objects. In PNAS, a team from the Technical University of Munich (TUM) describes a Nano-CT device that creates three-dimensional x-ray images at resolutions up to 100 nanometers. The first test application: Together with colleagues from the University of Kassel and Helmholtz-Zentrum Geesthacht the researchers analyzed the locomotory system of a velvet worm.

During a CT analysis, the object under investigation is x-rayed and a detector measures the respective amount of radiation absorbed from various angles....

Im Focus: Researchers Develop Data Bus for Quantum Computer

The quantum world is fragile; error correction codes are needed to protect the information stored in a quantum object from the deteriorating effects of noise. Quantum physicists in Innsbruck have developed a protocol to pass quantum information between differently encoded building blocks of a future quantum computer, such as processors and memories. Scientists may use this protocol in the future to build a data bus for quantum computers. The researchers have published their work in the journal Nature Communications.

Future quantum computers will be able to solve problems where conventional computers fail today. We are still far away from any large-scale implementation,...

Im Focus: Wrinkles give heat a jolt in pillared graphene

Rice University researchers test 3-D carbon nanostructures' thermal transport abilities

Pillared graphene would transfer heat better if the theoretical material had a few asymmetric junctions that caused wrinkles, according to Rice University...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

Ecology Across Borders: International conference brings together 1,500 ecologists

15.11.2017 | Event News

Road into laboratory: Users discuss biaxial fatigue-testing for car and truck wheel

15.11.2017 | Event News

#Berlin5GWeek: The right network for Industry 4.0

30.10.2017 | Event News

 
Latest News

NASA detects solar flare pulses at Sun and Earth

17.11.2017 | Physics and Astronomy

NIST scientists discover how to switch liver cancer cell growth from 2-D to 3-D structures

17.11.2017 | Health and Medicine

The importance of biodiversity in forests could increase due to climate change

17.11.2017 | Studies and Analyses

VideoLinks
B2B-VideoLinks
More VideoLinks >>>