For instance, imagine typing in “Who starred in the film Casablanca?” The search engine would respond with “Humphrey Bogart and Ingrid Bergman.”
Try asking a more nuanced question, such as “What do Americans think of universal health care?” A search engine will create a report indicating trends in opinion based on what has been posted to the Web.
Search engines may eventually be used to conduct polling and even help sort fact from fiction, said Meng, who is helping to make such possibilities a reality, both through his research and as president of a company called Webscalers.
The way Meng sees it, big search engines such as Google and Yahoo are fundamentally flawed. The Web has two parts: the surface Web and the deep Web. The surface Web is made up of perhaps 60 billion pages. The deep Web, at some 900 billion pages, is about 15 times larger.
Google, which relies on a “crawler” to examine pages and catalog them for future searches, can search about 20 billion pages. Web crawlers follow links to reach pages and often miss content that isn’t linked to any other page or is in some way “hidden.”
Meng, along with researchers at the University of Illinois at Chicago and the University of Louisiana at Lafayette, has helped pioneer large-scale metasearch-engine technology that harnesses the power of small search engines to come up with results that are more accurate and more complete.
“Most of the pages on the deep Web aren’t directly ‘crawlable.’ We want to connect to small search engines and reach the deep Web,” he said. “That’s the idea. Many people have the misconception that Google can search everything, and if it’s not there it doesn’t exist. But we should be able to retrieve many times more than what Google can search.”
Not only can a metasearch engine probe deeper, it can also offer the latest information.
“In principle,” Meng said, “small guys are much better able to maintain the freshness of their data. Google has a program to ‘crawl’ all over the world. Depending on when the crawler has last visited your server, there’s a delay of days or weeks before a new page will show up in that search. We can get fresher results.”The concept is not new. In fact, the first metasearch engine was built in 1994.
The Web has millions of search engines at businesses, universities, newspapers and other organizations. Since 1997, and with continued funding from the National Science Foundation, Meng and his collaborators have found ways to run queries across multiple search engines and sort through the results.
Webscalers is based in the Start-Up Suite at Binghamton University’s Innovative Technologies Complex, which is home to several young companies that have their roots in faculty inventions.
“If the Web keeps on growing, a company like Google may run out of resources to crawl all of those pages,” said Vijay V. Raghavan, vice president of Webscalers and a faculty member at the University of Louisiana at Lafayette. “We won’t have that problem. We will scale much better.”
Webscalers’ technology could be useful for large organizations with many divisions. For example, Webscalers has developed a prototype that would allow a search of all 64 campuses in the State University of New York system as well as SUNY’s central administration.
“People can use it to find collaborators,” Meng said. “It could also help prospective students find programs they’re interested in.”
The technology could be adapted to large companies or even the government, Meng said.
Challenges for large-scale metasearch engines include determining which search engines are the best for a given query, automating the interaction with search engines as well as organizing the search results.
Meng hopes to build a grand metasearch engine one day that would integrate all of the 1 million small search engines into a single system. “There are still a lot of significant challenges in creating a system of such magnitude,” he said, “but I am optimistic that such a metasearch engine can be built.”Try out the concept online
• Webscalers also offers MySearchView, a system that allows any user to create his or her own metasearch engine just by checking off a few options at www.mysearchview.com
PhoxTroT: Optical Interconnect Technologies Revolutionized Data Centers and HPC Systems
11.12.2017 | Fraunhofer-Institut für Zuverlässigkeit und Mikrointegration IZM
Rules for superconductivity mirrored in 'excitonic insulator'
08.12.2017 | Rice University
Tiny pores at a cell's entryway act as miniature bouncers, letting in some electrically charged atoms--ions--but blocking others. Operating as exquisitely sensitive filters, these "ion channels" play a critical role in biological functions such as muscle contraction and the firing of brain cells.
To rapidly transport the right ions through the cell membrane, the tiny channels rely on a complex interplay between the ions and surrounding molecules,...
The miniaturization of the current technology of storage media is hindered by fundamental limits of quantum mechanics. A new approach consists in using so-called spin-crossover molecules as the smallest possible storage unit. Similar to normal hard drives, these special molecules can save information via their magnetic state. A research team from Kiel University has now managed to successfully place a new class of spin-crossover molecules onto a surface and to improve the molecule’s storage capacity. The storage density of conventional hard drives could therefore theoretically be increased by more than one hundred fold. The study has been published in the scientific journal Nano Letters.
Over the past few years, the building blocks of storage media have gotten ever smaller. But further miniaturization of the current technology is hindered by...
With innovative experiments, researchers at the Helmholtz-Zentrums Geesthacht and the Technical University Hamburg unravel why tiny metallic structures are extremely strong
Light-weight and simultaneously strong – porous metallic nanomaterials promise interesting applications as, for instance, for future aeroplanes with enhanced...
An interdisciplinary group of researchers interfaced individual bacteria with a computer to build a hybrid bio-digital circuit - Study published in Nature Communications
Scientists at the Institute of Science and Technology Austria (IST Austria) have managed to control the behavior of individual bacteria by connecting them to a...
Physicists in the Laboratory for Attosecond Physics (run jointly by LMU Munich and the Max Planck Institute for Quantum Optics) have developed an attosecond electron microscope that allows them to visualize the dispersion of light in time and space, and observe the motions of electrons in atoms.
The most basic of all physical interactions in nature is that between light and matter. This interaction takes place in attosecond times (i.e. billionths of a...
11.12.2017 | Event News
08.12.2017 | Event News
07.12.2017 | Event News
11.12.2017 | Physics and Astronomy
11.12.2017 | Earth Sciences
11.12.2017 | Information Technology