Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Researchers Develop Techniques for Computing Google-Style Web Rankingsup to Five Times Faster

14.05.2003


Speed-up may make "topic-sensitive" page rankings feasible

Computer science researchers at Stanford University have developed several new techniques that together may make it possible to calculate Web page rankings as used in the Google search engine up to five times faster. The speed-ups to Google’s method may make it realistic to calculate page rankings personalized for an individual’s interests or customized to a particular topic.

The Stanford team includes graduate students Sepandar Kamvar and Taher Haveliwala, noted numerical analyst Gene Golub and computer science professor Christopher Manning. They will present their first paper at the Twelfth Annual World Wide Web Conference (WWW2003) in Budapest, Hungary, May 20-24, 2003. The work was supported by the National Science Foundation (NSF), an independent federal agency that supports fundamental research and education in all fields of science and engineering.

Computing PageRank, the ranking algorithm behind the Google search engine, for a billion Web pages can take several days. Google currently ranks and searches 3 billion Web pages. Each personalized or topic-sensitive ranking would require a separate multi-day computation, but the payoff would be less time spent wading through irrelevant search results. For example, searching a sports-specific Google site for "Giants" would give more importance to pages about the New York or San Francisco Giants and less importance to pages about Jack and the Beanstalk.

"This work is a wonderful example of how NSF support for basic computer science research, including applied mathematics and algorithm research, has impacts in daily life," said NSF program officer Maria Zemankova. In the mid-1990s, an NSF digital library project and an NSF graduate fellowship also supported Stanford graduate students Larry Page and Sergey Brin while they developed what would become the Google search engine.

To speed up PageRank, the Stanford team developed a trio of techniques in numerical linear algebra. First, in the WWW2003 paper, they describe so-called "extrapolation" methods, which make some assumptions about the Web’s link structure that aren’t true, but permit a quick and easy computation of PageRank. Because the assumptions aren’t true, the PageRank isn’t exactly correct, but it’s close and can be refined using the original PageRank algorithm. The Stanford researchers have shown that their extrapolation techniques can speed up PageRank by 50 percent in realistic conditions and by up to 300 percent under less realistic conditions.

A second paper describes an enhancement, called "BlockRank," which relies on a feature of the Web’s link structure-a feature that the Stanford team is among the first to investigate and exploit. Namely, they show that approximately 80 percent of the pages on any given Web site point to other pages on the same site. As a result, they can compute many single-site PageRanks, glue them together in an appropriate manner and use that as a starting point for the original PageRank algorithm. With this technique, they can realistically speed up the PageRank computation by 300 percent.

Finally, the team notes in a third paper that the rankings for some pages are calculated early in the PageRank process, while the rankings of many highly rated pages take much longer to compute. In a method called "Adaptive PageRank," they eliminate redundant computations associated with those pages whose PageRanks finish early. This speeds up the PageRank computation by up to 50 percent.

"Further speed-ups are possible when we use all these methods," Kamvar said. "Our preliminary experiments show that combining the methods will make the computation of PageRank up to a factor of five faster. However, there are still several issues to be solved. We’re closer to a topic-based PageRank than to a personalized ranking."

The complexities of a personalized ranking would require even greater speed-ups to the PageRank calculations. In addition, while a faster algorithm shortens computation time, the issue of storage remains. Because the results from a single PageRank computation on a few billion Web pages require several gigabytes of storage, saving a personalized PageRank for many individuals would rapidly consume vast amounts of storage. Saving a limited number of topic-specific PageRank calculations would be more practical.

The reason for the expensive computation and storage requirements lies in how PageRank generates the rankings that have led to Google’s popularity. Unlike page-ranking methods that rate each page separately, PageRank bases each page’s "importance" on the number and importance of pages that link to the page.

Therefore, PageRank must consider all pages at the same time and can’t easily omit pages that aren’t likely to be relevant to a topic. It also means that the faster method will not affect how quickly Google presents results to users’ searches, because the rankings are computed in advance and not at the time a search is requested.

The Stanford team’s conference paper and technical reports on enhancing the PageRank algorithm, as well as the original paper describing the PageRank method, are available on the Stanford Database Group’s Publication Server (http://dbpubs.stanford.edu/).

David Hart | National Science Foundation
Further information:
http://www.stanford.edu/~sdkamvar/research.html
http://www.www2003.org/
http://dbpubs.stanford.edu

More articles from Information Technology:

nachricht Cloud technology: Dynamic certificates make cloud service providers more secure
15.01.2018 | Technische Universität München

nachricht New discovery could improve brain-like memory and computing
10.01.2018 | University of Minnesota

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Artificial agent designs quantum experiments

On the way to an intelligent laboratory, physicists from Innsbruck and Vienna present an artificial agent that autonomously designs quantum experiments. In initial experiments, the system has independently (re)discovered experimental techniques that are nowadays standard in modern quantum optical laboratories. This shows how machines could play a more creative role in research in the future.

We carry smartphones in our pockets, the streets are dotted with semi-autonomous cars, but in the research laboratory experiments are still being designed by...

Im Focus: Scientists decipher key principle behind reaction of metalloenzymes

So-called pre-distorted states accelerate photochemical reactions too

What enables electrons to be transferred swiftly, for example during photosynthesis? An interdisciplinary team of researchers has worked out the details of how...

Im Focus: The first precise measurement of a single molecule's effective charge

For the first time, scientists have precisely measured the effective electrical charge of a single molecule in solution. This fundamental insight of an SNSF Professor could also pave the way for future medical diagnostics.

Electrical charge is one of the key properties that allows molecules to interact. Life itself depends on this phenomenon: many biological processes involve...

Im Focus: Paradigm shift in Paris: Encouraging an holistic view of laser machining

At the JEC World Composite Show in Paris in March 2018, the Fraunhofer Institute for Laser Technology ILT will be focusing on the latest trends and innovations in laser machining of composites. Among other things, researchers at the booth shared with the Aachen Center for Integrative Lightweight Production (AZL) will demonstrate how lasers can be used for joining, structuring, cutting and drilling composite materials.

No other industry has attracted as much public attention to composite materials as the automotive industry, which along with the aerospace industry is a driver...

Im Focus: Room-temperature multiferroic thin films and their properties

Scientists at Tokyo Institute of Technology (Tokyo Tech) and Tohoku University have developed high-quality GFO epitaxial films and systematically investigated their ferroelectric and ferromagnetic properties. They also demonstrated the room-temperature magnetocapacitance effects of these GFO thin films.

Multiferroic materials show magnetically driven ferroelectricity. They are attracting increasing attention because of their fascinating properties such as...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

10th International Symposium: “Advanced Battery Power – Kraftwerk Batterie” Münster, 10-11 April 2018

08.01.2018 | Event News

See, understand and experience the work of the future

11.12.2017 | Event News

Innovative strategies to tackle parasitic worms

08.12.2017 | Event News

 
Latest News

Let the good tubes roll

19.01.2018 | Materials Sciences

How cancer metastasis happens: Researchers reveal a key mechanism

19.01.2018 | Health and Medicine

Meteoritic stardust unlocks timing of supernova dust formation

19.01.2018 | Physics and Astronomy

VideoLinks
B2B-VideoLinks
More VideoLinks >>>