Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

IBM Research is developing an enterprise-class anti-spam filter

25.08.2004


Spam is a massive problem - it currently accounts for between 1/3 and 1/2 of all emails and costs companies billions of dollars as the result of lower productivity, loss of legitimate messages and the need for increased bandwidth and storage. In a bid to try solve the problem, IBM has brought together scientists from different areas of research division to develop an enterprise anti-spam filtering system which combines several different filtering technologies to create the ultimate anti-spam system. For example one of the spam filters - Chung-Kwei - is a pattern-discovery-based system which uses an algorithm developed by life sciences researchers focused tackling computational biology challenges such as gene finding and protein annotation. By itself, Chung-Kwei detected 96.56 percent of spam messages with just a .066 percent false positive rate during tests conducted in IBM’s labs. By combining Chung-Kwei with the other spam filtering techniques, IBM researchers have created SpamGuru - a prototype anti-spam system which they believe has the potential to eliminate virtually all spam.

SpamGuru: An Enterprise Anti-Spam Filtering System

IBM Research is developing an enterprise-class anti-spam filter as part of our overall strategy of attacking the Spam problem on multiple fronts. Our anti-spam filter, SpamGuru, mirrors this philosophy by incorporating several different filtering technologies and intelligently combining their output to produce a single spamminess rating or score for each incoming message. The use of multiple algorithms improves the system’s effectiveness and makes it more difficult for spammers to attack. While a spammer may defeat any single algorithm, SpamGuru can rely on its remaining algorithms to maintain a high-degree of effectiveness.



SpamGuru’s filtering architecture uses multiple classification algorithms which are integrated into a single classification pipeline. SpamGuru’s pipeline allows it to benefit from multiple classifiers with minimum extra computational cost. SpamGuru’s classification technologies include spoof detection, Bayesian filtering, plagiarism detection, automatically generated white- and black-lists, and Chung-Kwei, a novel technique that uses advanced pattern-matching algorithms developed by IBM’s bioinformatics group.

Chung-Kwei: a Pattern-discovery-based System for the Automatic Identification of Unsolicited E-mail Messages (SPAM)

Chung-Kwei is a system that we developed recently for the analysis of electronic mail and the automatic identification and tagging of unsolicited messages (=spam). The underlying method uses pattern-discovery and has its underpinnings in a generic approach that has been behind successful solutions we developed for tackling computational biology problems such as gene finding and protein annotation. Chung-Kwei can be trained very quickly using a body of known spam/white messages and can do so without interrupting the ongoing classification of incoming e-mail. The prototype system, that we developed by training on a repository of 87,000 spam and white messages, achieved a sensitivity of 96.56% with a false positive rate of 0.066%, or one-in-six-thousand messages. In terms of speed, the Chung-Kwei prototype is capable of classifying approximately 200 messages per second, on a 2.2 GHz Intel-Pentium platform.

Christine Paulus | IBM
Further information:
http://www.ibm.com

More articles from Information Technology:

nachricht Fingerprints of quantum entanglement
16.02.2018 | University of Vienna

nachricht Simple in the Cloud: The digitalization of brownfield systems made easy
07.02.2018 | Deutsches Forschungszentrum für Künstliche Intelligenz GmbH, DFKI

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Demonstration of a single molecule piezoelectric effect

Breakthrough provides a new concept of the design of molecular motors, sensors and electricity generators at nanoscale

Researchers from the Institute of Organic Chemistry and Biochemistry of the CAS (IOCB Prague), Institute of Physics of the CAS (IP CAS) and Palacký University...

Im Focus: Hybrid optics bring color imaging using ultrathin metalenses into focus

For photographers and scientists, lenses are lifesavers. They reflect and refract light, making possible the imaging systems that drive discovery through the microscope and preserve history through cameras.

But today's glass-based lenses are bulky and resist miniaturization. Next-generation technologies, such as ultrathin cameras or tiny microscopes, require...

Im Focus: Stem cell divisions in the adult brain seen for the first time

Scientists from the University of Zurich have succeeded for the first time in tracking individual stem cells and their neuronal progeny over months within the intact adult brain. This study sheds light on how new neurons are produced throughout life.

The generation of new nerve cells was once thought to taper off at the end of embryonic development. However, recent research has shown that the adult brain...

Im Focus: Interference as a new method for cooling quantum devices

Theoretical physicists propose to use negative interference to control heat flow in quantum devices. Study published in Physical Review Letters

Quantum computer parts are sensitive and need to be cooled to very low temperatures. Their tiny size makes them particularly susceptible to a temperature...

Im Focus: Autonomous 3D scanner supports individual manufacturing processes

Let’s say the armrest is broken in your vintage car. As things stand, you would need a lot of luck and persistence to find the right spare part. But in the world of Industrie 4.0 and production with batch sizes of one, you can simply scan the armrest and print it out. This is made possible by the first ever 3D scanner capable of working autonomously and in real time. The autonomous scanning system will be on display at the Hannover Messe Preview on February 6 and at the Hannover Messe proper from April 23 to 27, 2018 (Hall 6, Booth A30).

Part of the charm of vintage cars is that they stopped making them long ago, so it is special when you do see one out on the roads. If something breaks or...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

2nd International Conference on High Temperature Shape Memory Alloys (HTSMAs)

15.02.2018 | Event News

Aachen DC Grid Summit 2018

13.02.2018 | Event News

How Global Climate Policy Can Learn from the Energy Transition

12.02.2018 | Event News

 
Latest News

Fingerprints of quantum entanglement

16.02.2018 | Information Technology

'Living bandages': NUST MISIS scientists develop biocompatible anti-burn nanofibers

16.02.2018 | Health and Medicine

Hubble sees Neptune's mysterious shrinking storm

16.02.2018 | Physics and Astronomy

VideoLinks
Science & Research
Overview of more VideoLinks >>>