Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

IBM Research is developing an enterprise-class anti-spam filter

25.08.2004


Spam is a massive problem - it currently accounts for between 1/3 and 1/2 of all emails and costs companies billions of dollars as the result of lower productivity, loss of legitimate messages and the need for increased bandwidth and storage. In a bid to try solve the problem, IBM has brought together scientists from different areas of research division to develop an enterprise anti-spam filtering system which combines several different filtering technologies to create the ultimate anti-spam system. For example one of the spam filters - Chung-Kwei - is a pattern-discovery-based system which uses an algorithm developed by life sciences researchers focused tackling computational biology challenges such as gene finding and protein annotation. By itself, Chung-Kwei detected 96.56 percent of spam messages with just a .066 percent false positive rate during tests conducted in IBM’s labs. By combining Chung-Kwei with the other spam filtering techniques, IBM researchers have created SpamGuru - a prototype anti-spam system which they believe has the potential to eliminate virtually all spam.

SpamGuru: An Enterprise Anti-Spam Filtering System

IBM Research is developing an enterprise-class anti-spam filter as part of our overall strategy of attacking the Spam problem on multiple fronts. Our anti-spam filter, SpamGuru, mirrors this philosophy by incorporating several different filtering technologies and intelligently combining their output to produce a single spamminess rating or score for each incoming message. The use of multiple algorithms improves the system’s effectiveness and makes it more difficult for spammers to attack. While a spammer may defeat any single algorithm, SpamGuru can rely on its remaining algorithms to maintain a high-degree of effectiveness.



SpamGuru’s filtering architecture uses multiple classification algorithms which are integrated into a single classification pipeline. SpamGuru’s pipeline allows it to benefit from multiple classifiers with minimum extra computational cost. SpamGuru’s classification technologies include spoof detection, Bayesian filtering, plagiarism detection, automatically generated white- and black-lists, and Chung-Kwei, a novel technique that uses advanced pattern-matching algorithms developed by IBM’s bioinformatics group.

Chung-Kwei: a Pattern-discovery-based System for the Automatic Identification of Unsolicited E-mail Messages (SPAM)

Chung-Kwei is a system that we developed recently for the analysis of electronic mail and the automatic identification and tagging of unsolicited messages (=spam). The underlying method uses pattern-discovery and has its underpinnings in a generic approach that has been behind successful solutions we developed for tackling computational biology problems such as gene finding and protein annotation. Chung-Kwei can be trained very quickly using a body of known spam/white messages and can do so without interrupting the ongoing classification of incoming e-mail. The prototype system, that we developed by training on a repository of 87,000 spam and white messages, achieved a sensitivity of 96.56% with a false positive rate of 0.066%, or one-in-six-thousand messages. In terms of speed, the Chung-Kwei prototype is capable of classifying approximately 200 messages per second, on a 2.2 GHz Intel-Pentium platform.

Christine Paulus | IBM
Further information:
http://www.ibm.com

More articles from Information Technology:

nachricht Researchers build transistor-like gate for quantum information processing -- with qudits
17.07.2019 | Purdue University

nachricht New DFG Research Group "Metrology for THz Communications"
17.07.2019 | Technische Universität Braunschweig

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Better thermal conductivity by adjusting the arrangement of atoms

Adjusting the thermal conductivity of materials is one of the challenges nanoscience is currently facing. Together with colleagues from the Netherlands and Spain, researchers from the University of Basel have shown that the atomic vibrations that determine heat generation in nanowires can be controlled through the arrangement of atoms alone. The scientists will publish the results shortly in the journal Nano Letters.

In the electronics and computer industry, components are becoming ever smaller and more powerful. However, there are problems with the heat generation. It is...

Im Focus: First-ever visualizations of electrical gating effects on electronic structure

Scientists have visualised the electronic structure in a microelectronic device for the first time, opening up opportunities for finely-tuned high performance electronic devices.

Physicists from the University of Warwick and the University of Washington have developed a technique to measure the energy and momentum of electrons in...

Im Focus: Megakaryocytes act as „bouncers“ restraining cell migration in the bone marrow

Scientists at the University Würzburg and University Hospital of Würzburg found that megakaryocytes act as “bouncers” and thus modulate bone marrow niche properties and cell migration dynamics. The study was published in July in the Journal “Haematologica”.

Hematopoiesis is the process of forming blood cells, which occurs predominantly in the bone marrow. The bone marrow produces all types of blood cells: red...

Im Focus: Artificial neural network resolves puzzles from condensed matter physics: Which is the perfect quantum theory?

For some phenomena in quantum many-body physics several competing theories exist. But which of them describes a quantum phenomenon best? A team of researchers from the Technical University of Munich (TUM) and Harvard University in the United States has now successfully deployed artificial neural networks for image analysis of quantum systems.

Is that a dog or a cat? Such a classification is a prime example of machine learning: artificial neural networks can be trained to analyze images by looking...

Im Focus: Extremely hard yet metallically conductive: Bayreuth researchers develop novel material with high-tech prospects

An international research group led by scientists from the University of Bayreuth has produced a previously unknown material: Rhenium nitride pernitride. Thanks to combining properties that were previously considered incompatible, it looks set to become highly attractive for technological applications. Indeed, it is a super-hard metallic conductor that can withstand extremely high pressures like a diamond. A process now developed in Bayreuth opens up the possibility of producing rhenium nitride pernitride and other technologically interesting materials in sufficiently large quantity for their properties characterisation. The new findings are presented in "Nature Communications".

The possibility of finding a compound that was metallically conductive, super-hard, and ultra-incompressible was long considered unlikely in science. It was...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

2nd International Conference on UV LED Technologies & Applications – ICULTA 2020 | Call for Abstracts

24.06.2019 | Event News

SEMANTiCS 2019 brings together industry leaders and data scientists in Karlsruhe

29.04.2019 | Event News

Revered mathematicians and computer scientists converge with 200 young researchers in Heidelberg!

17.04.2019 | Event News

 
Latest News

Heat flow through single molecules detected

19.07.2019 | Physics and Astronomy

Heat transport through single molecules

19.07.2019 | Physics and Astronomy

Welcome Committee for Comets

19.07.2019 | Earth Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>