Ending Spam

Ending Spam
Bayesian Content Filtering and the Art of Statistical Language Classification
Jonathan A. Zdziarski
July 2005, 312 pp.

Through considerable research, creative minds have invented clever new ways to fight spam in all its nefarious forms. This landmark title describes, in depth, how statistical filtering is being used by next generation spam filters to identify and filter spam. Zdziarski explains how spam filtering works and how language classification and machine learning combine to produce remarkably accurate spam filters. Readers gain a complete understanding of the mathematical approaches used in today's spam filters, decoding, tokenization, the use of various algorithms (including Bayesian analysis and Markovian discrimination), and the benefits of using open-source solutions to end spam. Interviews with the creators of many of the best spam filters provide further insight into the anti-spam crusade.

Author Bio 

Jonathan A. Zdziarski has been fighting spam for eight years, and
has spent a significant portion of the past two years working on the next
generation spam filter DSPAM, with up to 99.985% accuracy. Zdziarski
lectures widely on the topic of spam.

Table of contents 


PART I: An Introduction to Spam Filtering
Chapter 1: The History of Spam
Chapter 2: Historical Approaches to Fighting Spam
Chapter 3: Language Classification Concepts
Chapter 4: Statistical Filtering Fundamentals

PART II: Fundamentals of Statistical Filtering
Chapter 5: Decoding: Uncombobulating Messages
Chapter 6: Tokenization: The Building Blocks of Spam
Chapter 7: The Low-Down Dirty Tricks of Spammers
Chapter 8: Data Storage for a Zillion Records
Chapter 9: Scaling in Large Environments

PART III: Advanced Concepts of Statistical Filtering
Chapter 10: Testing Theory
Chapter 11: Concept Identification: Advanced Tokenization
Chapter 12: Fifth-Order Markovian Discrimination
Chapter 13: Intelligent Feature Set Reduction
Chapter 14: Collaborative Algorithms

Appendix: Shining Examples of Filtering


A detailed table of contents is available here.


"The real beauty . . . is that it provides detailed explanations of filtering techniques without overwhelming or boring casual readers . . . The all-encompassing nature of the book should therefore earn it a space on the shelf of anyone that wants to actively contribute to the reduction of spam."
Dr. Dobb’s Journal, October 2006 (Read more)

"The only book I have seen that discusses the underlying science of spam filtering."
PC Update, April 2006 (Read more)

Included in roundup of infosec and related books: "This book reads like a doctorate-level thesis."
Security.ITWorld.com, December 2, 2005 (Read more)

"For those who are seriously involved in managing and developing spam filtering, the book does provide very useful advice, pointers, and research."
Internet Review Project (Read more)

"Besides giving a pretty insightful view on spammer techniques, the book’s value is the vast coverage of mathematical approaches to spam filtering."
Insecure Magazine, October 2005 (Read more)

Book giveaway contest for readers
Free Software Magazine, November 7, 2005 (Read more)

"The book is accessible to any reader with a computer science background and is essential reading for anyone wanting to understand Bayesian spam filtering."
Virus Bulletin (subscription only), October 2005 (Read more)

Blurb on new version of DSPAM includes mention of Ending Spam
Slashdot, October 17, 2005 (Read more)

"You finish the book feeling good that there are people like Zdziarski fighting the good fight."
Free Software Magazine, October 2005 (Read more)

Blog of Ferris Research email analyst: "An excellent primer on spam, spammers, and spam fighting."
Richi’Blog, October 25, 2005 (Read more)

"Even if you’re just looking for an introduction to spam, this book is worth picking up."
TipMonkeys, August 19, 2005 (Read more)

Interview with Jonathan Zdziarski
GeekNews, October 2, 2005 (Read more)

Excerpt posted
Sys-Con France, October 3, 2005 (Read more)

Jonathan Zdziarski heavily quoted in two articles about open source anti-spam solutions
Channel Insider, "Slamming Spam, For Free," September 21, 2005 (Read more) and "Misplaced Caution About Open-Source Anti-Spam?," September 23, 2005 (Read more)

"Includes an excellent section on spammer tricks and how they get past filters as well as what to do about it. This section alone makes the book worth the price... a highly recommended read for anyone in charge of controlling spam in a corporate environment."
Midwest Book Review, September 2005(Read more)

"Not only enjoyable but actually captivating.”
Linux Magazine, October 2005
(Read more)

"There’s no one more than me who hopes it’s only a matter of time before e-mail spam disappears -– and books like the recently released [Ending Spam] by anti-spam expert Jonathan Zdziarski will make this happen."
Editor-in-chief’s column in Dr. Dobb’s Journal, September 2005
(Read more)

"Leads the charge against what has become a very significant challenge to both productivity and sanity by explaining how it was spawned, its anatomy and physiognomy, how it crawled into our pristine PCs, and how, as responsible and gleeful citizens, we can torch it."
Book News, September 2005
(Read more)

Interview with Jonathan Zdziarski from reader-submitted questions
Slashdot, August 30, 2005
(Read more)

"This book is down, and dirty, loaded with information, and will make your head-hurt (in a good way)."
MyMac, August 29, 2005
(Read more)

"Does a good job of addressing advanced, complicated issues, but putting it in terms that readers with an ounce of computer knowledge and experience can grasp."
Netsecurity.about.com, August 29, 2005
(Read more)

"Highly recommended read for anyone in charge of controlling spam in a corporate environment as well as on their own system."
Readers Preference (Read more)

"The first book explaining the fine details of the theoretical models and machine-learning algorithms implemented in these filters... Zdziarski has made another significant contribution (after DSPAM) to the anti-spam community."
Slashdot, August 15, 2005
(Read more)

Mention of Zdziarski finding possible flaw in member area of Verizon Wireless website
eWeek, August 15, 2005
(Read more)

Author Jonathan Zdziarski interviewed
Computer Outlook Radio, August 9, 2005
(Listen to the interview)

"A remarkably lucid explanation of concepts and tools that many of us may never have heard of before."
Linux Users of Victoria (Australia), August 9, 2005
(Read more)

Book featured on Washington, D.C.-area NPR program
The Computer Guys, August 2, 2005
(Read more)

Posted link to book on "Plan For Spam" page
Paul Graham, programming & spam filter pioneer
(Read more)

Book is featured anti-spam resource
(Read more)

5 stars out of 5: Ending Spam "does a great job educating us on logic and thought taken to combat this SPAM blight on the Internet."
MacCompanion, August 2005 (Read more)

Zdziarski interview podcast "about his views as a highly regarded programmer and researcher, the latest cures for spam and how spam affects SOHO, home-office and small business owners."
Kickstart News, July 31, 2005
(Read more)

Zdziarski on for an hour discussing DSPAM and new book
Online Tonight, July 25, 2005 (Read more)

"IT managers who want a better understanding of how anti-spam products work should shell out the $39.95 price of this 312-page book at once."
eWeek, July 25, 2005
(Read more)

"It’s one of the better books on the details of spam and spam filtering...
if you're more of a ‘I need to have deep knowledge of how does this
actually work’ person, go with Zdziarski first."
CRM114 Discriminator, July 18, 2005 (Read more)

"The clearest and most erudite treatise on spam filtering to date."
Kickstart News, July 17, 2005 (Read more)

"The author explains everything in simple and entertaining language... I'm
a little hesitant to call this book entertaining, although it actually
is... because saying that might give the impression that there is more
fluff than substance, and that's not the case. There is a lot of
substance here, both in theory and in practical advice."
A.P. Lawrence, July 16, 2005 (Read more)

"If you’re looking for a primer on how the anti-spam battle is fought,
you can’t do much better."
InfoWorld, Summer Reading List for the Geek Crowd, July 14, 2005 (Read more)

Interview with author about anti-spam techniques
Network World, July 4, 2005 (Full interview)

"Has a permanent place on my bookshelf, because I find it inspirational."
Edmonton Linux User Group, July 2005
(Read more)

Book excerpt posted
IT World, June 27, 2005 (Read excertp)

Book announcement posted
ITSecurity.com, June 21, 2005 (Read announcement)