Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Computer Scientists from Saarbrücken Search Through Large Datasets using "Good Trojans"

13.09.2010
Social networks, search engines, digital archives, and several global-scale internet companies host very large data collections.

In order to search through this data companies like Facebook, Ebay, Yahoo, and Twitter use the freely available Hadoop software - a variant of MapReduce originally proposed by Google. Several database experts, however, criticize Hadoop for being inefficient.

Computer scientists from Saarland University (Germany) are now proposing a new system coined Hadoop++. It allows users to search through very large datasets much faster than before. Hadoop++ improves over Hadoop by up to a factor of 20.

Internet companies need to process data volumes on the order of millions of Gigabytes (Petabytes) on a daily basis. In order to search through this data effectively, Google proposed the MapReduce programming model. MapReduce divides the input data into smaller chunks that are then distributed over a large network of machines and processed in parallel. The open-source counterpart of MapReduce is called Hadoop. Even though Google holds a patent on MapReduce, it granted a license to Hadoop. Therefore, Hadoop may still be used by companies free of charge. However, "Database experts who are fluent in SQL consider MapReduce a major step backwards towards the database stone age", explains Jens Dittrich, Professor of Information Systems at Saarland University. He adds, "MapReduce disregards considerable wisdom from database research. As a consequence MapReduce is often slow and inefficient."

Despite this criticism, over the past years Hadoop has gained considerable attention from both industry and academia. Hadoop is very popular among programmers throughout the world. Professor Dittrich explains why, "The reason is its ease of use: the user neither has to learn a complex database language nor a data model. Furthermore, Hadoop is relatively easy to administrate. In summary, Hadoop allows even database-illiterate people to search through billions of records on very large computer clusters."

However, this comes at a price: "When compared to modern relational database management systems, Hadoop is just too slow." Therefore the researcher and his team at Saarland University have developed a new system, coined Hadoop++. It aims to eliminate the performance deficiencies encountered in Hadoop. The creativity of the new approach lies in how the problem is tackled: Hadoop++ works similarly to a "trojan", i.e. a computer virus which infects a computer system clandestinely and may then cause considerable harm. In contrast to these bad trojans, Hadoop++ injects hidden code into a system in order to "heal" it, ie. dynamically accelerating the underlying Hadoop. Professor Dittrich emphasizes, "One could say that Hadoop++ is a good trojan".

The Saarbrücken approach has the considerable advantage that Hadoop's tested code base does not have to be modified and retested. Thus, Hadoop++ avoids complex changes to a working system and unforeseen consequences. Hadoop++ will be presented at this year's International Conference on Very Large Databases (VLDB) - one of the world's most prestigious database conferences to be held in Singapore from September 13-17.

Background

There have been heated discussions about the pros and cons of MapReduce/Hadoop when compared to traditional database management systems. This discussion was led by database professors in the US. Please refer to the links below for details. Recently there have been attempts to improve the runtime efficiency of MapReduce. However, as Professor Dittrich explains, "those attempts could not really marry MapReduce with database technology."

Press Pictures: www.uni-saarland.de/pressefotos

For questions, contact:
Jens Dittrich
Professor of Information Systems
Saarland University
Tel. (+49) 681 302 70141
Weitere Informationen:
http://infosys.cs.uni-saarland.de/hadoop++.php
http://hadoop.apache.org/
http://www.mapreduce.org/
http://www.vldb2010.org

Friederike Meyer zu Tittingdorf | idw
Further information:
http://www.uni-saarland.de

More articles from Information Technology:

nachricht Supercomputing the emergence of material behavior
18.05.2018 | University of Texas at Austin, Texas Advanced Computing Center

nachricht Keeping a Close Eye on Ice Loss
18.05.2018 | Alfred-Wegener-Institut, Helmholtz-Zentrum für Polar- und Meeresforschung

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Explanation for puzzling quantum oscillations has been found

So-called quantum many-body scars allow quantum systems to stay out of equilibrium much longer, explaining experiment | Study published in Nature Physics

Recently, researchers from Harvard and MIT succeeded in trapping a record 53 atoms and individually controlling their quantum state, realizing what is called a...

Im Focus: Dozens of binaries from Milky Way's globular clusters could be detectable by LISA

Next-generation gravitational wave detector in space will complement LIGO on Earth

The historic first detection of gravitational waves from colliding black holes far outside our galaxy opened a new window to understanding the universe. A...

Im Focus: Entangled atoms shine in unison

A team led by Austrian experimental physicist Rainer Blatt has succeeded in characterizing the quantum entanglement of two spatially separated atoms by observing their light emission. This fundamental demonstration could lead to the development of highly sensitive optical gradiometers for the precise measurement of the gravitational field or the earth's magnetic field.

The age of quantum technology has long been heralded. Decades of research into the quantum world have led to the development of methods that make it possible...

Im Focus: Computer-Designed Customized Regenerative Heart Valves

Cardiovascular tissue engineering aims to treat heart disease with prostheses that grow and regenerate. Now, researchers from the University of Zurich, the Technical University Eindhoven and the Charité Berlin have successfully implanted regenerative heart valves, designed with the aid of computer simulations, into sheep for the first time.

Producing living tissue or organs based on human cells is one of the main research fields in regenerative medicine. Tissue engineering, which involves growing...

Im Focus: Light-induced superconductivity under high pressure

A team of scientists of the Max Planck Institute for the Structure and Dynamics of Matter (MPSD) at the Center for Free-Electron Laser Science in Hamburg investigated optically-induced superconductivity in the alkali-doped fulleride K3C60under high external pressures. This study allowed, on one hand, to uniquely assess the nature of the transient state as a superconducting phase. In addition, it unveiled the possibility to induce superconductivity in K3C60 at temperatures far above the -170 degrees Celsius hypothesized previously, and rather all the way to room temperature. The paper by Cantaluppi et al has been published in Nature Physics.

Unlike ordinary metals, superconductors have the unique capability of transporting electrical currents without any loss. Nowadays, their technological...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Save the date: Forum European Neuroscience – 07-11 July 2018 in Berlin, Germany

02.05.2018 | Event News

Invitation to the upcoming "Current Topics in Bioinformatics: Big Data in Genomics and Medicine"

13.04.2018 | Event News

Unique scope of UV LED technologies and applications presented in Berlin: ICULTA-2018

12.04.2018 | Event News

 
Latest News

Supersonic waves may help electronics beat the heat

18.05.2018 | Power and Electrical Engineering

Keeping a Close Eye on Ice Loss

18.05.2018 | Information Technology

CrowdWater: An App for Flood Research

18.05.2018 | Information Technology

VideoLinks
Science & Research
Overview of more VideoLinks >>>