In order to search through this data companies like Facebook, Ebay, Yahoo, and Twitter use the freely available Hadoop software - a variant of MapReduce originally proposed by Google. Several database experts, however, criticize Hadoop for being inefficient.
Computer scientists from Saarland University (Germany) are now proposing a new system coined Hadoop++. It allows users to search through very large datasets much faster than before. Hadoop++ improves over Hadoop by up to a factor of 20.
Internet companies need to process data volumes on the order of millions of Gigabytes (Petabytes) on a daily basis. In order to search through this data effectively, Google proposed the MapReduce programming model. MapReduce divides the input data into smaller chunks that are then distributed over a large network of machines and processed in parallel. The open-source counterpart of MapReduce is called Hadoop. Even though Google holds a patent on MapReduce, it granted a license to Hadoop. Therefore, Hadoop may still be used by companies free of charge. However, "Database experts who are fluent in SQL consider MapReduce a major step backwards towards the database stone age", explains Jens Dittrich, Professor of Information Systems at Saarland University. He adds, "MapReduce disregards considerable wisdom from database research. As a consequence MapReduce is often slow and inefficient."
Despite this criticism, over the past years Hadoop has gained considerable attention from both industry and academia. Hadoop is very popular among programmers throughout the world. Professor Dittrich explains why, "The reason is its ease of use: the user neither has to learn a complex database language nor a data model. Furthermore, Hadoop is relatively easy to administrate. In summary, Hadoop allows even database-illiterate people to search through billions of records on very large computer clusters."
However, this comes at a price: "When compared to modern relational database management systems, Hadoop is just too slow." Therefore the researcher and his team at Saarland University have developed a new system, coined Hadoop++. It aims to eliminate the performance deficiencies encountered in Hadoop. The creativity of the new approach lies in how the problem is tackled: Hadoop++ works similarly to a "trojan", i.e. a computer virus which infects a computer system clandestinely and may then cause considerable harm. In contrast to these bad trojans, Hadoop++ injects hidden code into a system in order to "heal" it, ie. dynamically accelerating the underlying Hadoop. Professor Dittrich emphasizes, "One could say that Hadoop++ is a good trojan".
The Saarbrücken approach has the considerable advantage that Hadoop's tested code base does not have to be modified and retested. Thus, Hadoop++ avoids complex changes to a working system and unforeseen consequences. Hadoop++ will be presented at this year's International Conference on Very Large Databases (VLDB) - one of the world's most prestigious database conferences to be held in Singapore from September 13-17.
There have been heated discussions about the pros and cons of MapReduce/Hadoop when compared to traditional database management systems. This discussion was led by database professors in the US. Please refer to the links below for details. Recently there have been attempts to improve the runtime efficiency of MapReduce. However, as Professor Dittrich explains, "those attempts could not really marry MapReduce with database technology."
Press Pictures: www.uni-saarland.de/pressefotosFor questions, contact:
Friederike Meyer zu Tittingdorf | idw
Information integration and artificial intelligence for better diagnosis and therapy decisions
24.05.2017 | Fraunhofer MEVIS - Institut für Bildgestützte Medizin
World's thinnest hologram paves path to new 3-D world
18.05.2017 | RMIT University
Staphylococcus aureus is a feared pathogen (MRSA, multi-resistant S. aureus) due to frequent resistances against many antibiotics, especially in hospital infections. Researchers at the Paul-Ehrlich-Institut have identified immunological processes that prevent a successful immune response directed against the pathogenic agent. The delivery of bacterial proteins with RNA adjuvant or messenger RNA (mRNA) into immune cells allows the re-direction of the immune response towards an active defense against S. aureus. This could be of significant importance for the development of an effective vaccine. PLOS Pathogens has published these research results online on 25 May 2017.
Staphylococcus aureus (S. aureus) is a bacterium that colonizes by far more than half of the skin and the mucosa of adults, usually without causing infections....
Physicists from the University of Würzburg are capable of generating identical looking single light particles at the push of a button. Two new studies now demonstrate the potential this method holds.
The quantum computer has fuelled the imagination of scientists for decades: It is based on fundamentally different phenomena than a conventional computer....
An international team of physicists has monitored the scattering behaviour of electrons in a non-conducting material in real-time. Their insights could be beneficial for radiotherapy.
We can refer to electrons in non-conducting materials as ‘sluggish’. Typically, they remain fixed in a location, deep inside an atomic composite. It is hence...
Two-dimensional magnetic structures are regarded as a promising material for new types of data storage, since the magnetic properties of individual molecular building blocks can be investigated and modified. For the first time, researchers have now produced a wafer-thin ferrimagnet, in which molecules with different magnetic centers arrange themselves on a gold surface to form a checkerboard pattern. Scientists at the Swiss Nanoscience Institute at the University of Basel and the Paul Scherrer Institute published their findings in the journal Nature Communications.
Ferrimagnets are composed of two centers which are magnetized at different strengths and point in opposing directions. Two-dimensional, quasi-flat ferrimagnets...
An Australian-Chinese research team has created the world's thinnest hologram, paving the way towards the integration of 3D holography into everyday...
24.05.2017 | Event News
23.05.2017 | Event News
22.05.2017 | Event News
26.05.2017 | Life Sciences
26.05.2017 | Life Sciences
26.05.2017 | Physics and Astronomy