Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

IBM Research is developing an enterprise-class anti-spam filter

25.08.2004


Spam is a massive problem - it currently accounts for between 1/3 and 1/2 of all emails and costs companies billions of dollars as the result of lower productivity, loss of legitimate messages and the need for increased bandwidth and storage. In a bid to try solve the problem, IBM has brought together scientists from different areas of research division to develop an enterprise anti-spam filtering system which combines several different filtering technologies to create the ultimate anti-spam system. For example one of the spam filters - Chung-Kwei - is a pattern-discovery-based system which uses an algorithm developed by life sciences researchers focused tackling computational biology challenges such as gene finding and protein annotation. By itself, Chung-Kwei detected 96.56 percent of spam messages with just a .066 percent false positive rate during tests conducted in IBM’s labs. By combining Chung-Kwei with the other spam filtering techniques, IBM researchers have created SpamGuru - a prototype anti-spam system which they believe has the potential to eliminate virtually all spam.

SpamGuru: An Enterprise Anti-Spam Filtering System

IBM Research is developing an enterprise-class anti-spam filter as part of our overall strategy of attacking the Spam problem on multiple fronts. Our anti-spam filter, SpamGuru, mirrors this philosophy by incorporating several different filtering technologies and intelligently combining their output to produce a single spamminess rating or score for each incoming message. The use of multiple algorithms improves the system’s effectiveness and makes it more difficult for spammers to attack. While a spammer may defeat any single algorithm, SpamGuru can rely on its remaining algorithms to maintain a high-degree of effectiveness.



SpamGuru’s filtering architecture uses multiple classification algorithms which are integrated into a single classification pipeline. SpamGuru’s pipeline allows it to benefit from multiple classifiers with minimum extra computational cost. SpamGuru’s classification technologies include spoof detection, Bayesian filtering, plagiarism detection, automatically generated white- and black-lists, and Chung-Kwei, a novel technique that uses advanced pattern-matching algorithms developed by IBM’s bioinformatics group.

Chung-Kwei: a Pattern-discovery-based System for the Automatic Identification of Unsolicited E-mail Messages (SPAM)

Chung-Kwei is a system that we developed recently for the analysis of electronic mail and the automatic identification and tagging of unsolicited messages (=spam). The underlying method uses pattern-discovery and has its underpinnings in a generic approach that has been behind successful solutions we developed for tackling computational biology problems such as gene finding and protein annotation. Chung-Kwei can be trained very quickly using a body of known spam/white messages and can do so without interrupting the ongoing classification of incoming e-mail. The prototype system, that we developed by training on a repository of 87,000 spam and white messages, achieved a sensitivity of 96.56% with a false positive rate of 0.066%, or one-in-six-thousand messages. In terms of speed, the Chung-Kwei prototype is capable of classifying approximately 200 messages per second, on a 2.2 GHz Intel-Pentium platform.

Christine Paulus | IBM
Further information:
http://www.ibm.com

More articles from Information Technology:

nachricht NASA CubeSat to test miniaturized weather satellite technology
10.11.2017 | NASA/Goddard Space Flight Center

nachricht New approach uses light instead of robots to assemble electronic components
08.11.2017 | The Optical Society

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Nanoparticles help with malaria diagnosis – new rapid test in development

The WHO reports an estimated 429,000 malaria deaths each year. The disease mostly affects tropical and subtropical regions and in particular the African continent. The Fraunhofer Institute for Silicate Research ISC teamed up with the Fraunhofer Institute for Molecular Biology and Applied Ecology IME and the Institute of Tropical Medicine at the University of Tübingen for a new test method to detect malaria parasites in blood. The idea of the research project “NanoFRET” is to develop a highly sensitive and reliable rapid diagnostic test so that patient treatment can begin as early as possible.

Malaria is caused by parasites transmitted by mosquito bite. The most dangerous form of malaria is malaria tropica. Left untreated, it is fatal in most cases....

Im Focus: A “cosmic snake” reveals the structure of remote galaxies

The formation of stars in distant galaxies is still largely unexplored. For the first time, astron-omers at the University of Geneva have now been able to closely observe a star system six billion light-years away. In doing so, they are confirming earlier simulations made by the University of Zurich. One special effect is made possible by the multiple reflections of images that run through the cosmos like a snake.

Today, astronomers have a pretty accurate idea of how stars were formed in the recent cosmic past. But do these laws also apply to older galaxies? For around a...

Im Focus: Visual intelligence is not the same as IQ

Just because someone is smart and well-motivated doesn't mean he or she can learn the visual skills needed to excel at tasks like matching fingerprints, interpreting medical X-rays, keeping track of aircraft on radar displays or forensic face matching.

That is the implication of a new study which shows for the first time that there is a broad range of differences in people's visual ability and that these...

Im Focus: Novel Nano-CT device creates high-resolution 3D-X-rays of tiny velvet worm legs

Computer Tomography (CT) is a standard procedure in hospitals, but so far, the technology has not been suitable for imaging extremely small objects. In PNAS, a team from the Technical University of Munich (TUM) describes a Nano-CT device that creates three-dimensional x-ray images at resolutions up to 100 nanometers. The first test application: Together with colleagues from the University of Kassel and Helmholtz-Zentrum Geesthacht the researchers analyzed the locomotory system of a velvet worm.

During a CT analysis, the object under investigation is x-rayed and a detector measures the respective amount of radiation absorbed from various angles....

Im Focus: Researchers Develop Data Bus for Quantum Computer

The quantum world is fragile; error correction codes are needed to protect the information stored in a quantum object from the deteriorating effects of noise. Quantum physicists in Innsbruck have developed a protocol to pass quantum information between differently encoded building blocks of a future quantum computer, such as processors and memories. Scientists may use this protocol in the future to build a data bus for quantum computers. The researchers have published their work in the journal Nature Communications.

Future quantum computers will be able to solve problems where conventional computers fail today. We are still far away from any large-scale implementation,...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

Event News

Ecology Across Borders: International conference brings together 1,500 ecologists

15.11.2017 | Event News

Road into laboratory: Users discuss biaxial fatigue-testing for car and truck wheel

15.11.2017 | Event News

#Berlin5GWeek: The right network for Industry 4.0

30.10.2017 | Event News

 
Latest News

Corporate coworking as a driver of innovation

22.11.2017 | Business and Finance

PPPL scientists deliver new high-resolution diagnostic to national laser facility

22.11.2017 | Physics and Astronomy

Quantum optics allows us to abandon expensive lasers in spectroscopy

22.11.2017 | Physics and Astronomy

VideoLinks
B2B-VideoLinks
More VideoLinks >>>