Friday, November 23, 2007
« Lariat.net the world's first wireless br... | Main | ITU Botnet Mitigation Toolkit »
The Future of Internet Immune Systems
The Future of Internet Immune Systems
Written by Cory Doctorow
11/19/2007
<http://www.internetevolution.com/author.asp?section_id=479&doc_id=139358&>

Bunhill Cemetery is just down the road from my flat in London. It’s a 
handsome old boneyard, a former plague pit (“Bone hill” -- as in, 
there are so many bones under there that the ground is actually kind 
of humped up into a hill). There are plenty of luminaries buried there 
-- John “Pilgrim’s Progress” Bunyan, William Blake, Daniel Defoe, and 
assorted Cromwells. But my favorite tomb is that of Thomas Bayes, the 
18th-century statistician for whom Bayesian filtering is named.

Bayesian filtering is plenty useful. Here’s a simple example of how 
you might use a Bayesian filter. First, get a giant load of non-spam 
emails and feed them into a Bayesian program that counts how many 
times each word in their vocabulary appears, producing a statistical 
breakdown of the word-frequency in good emails.

Then, point the filter at a giant load of spam (if you’re having a 
hard time getting a hold of one, I have plenty to spare), and count 
the words in it. Now, for each new message that arrives in your inbox, 
have the filter count the relative word-frequencies and make a 
statistical prediction about whether the new message is spam or not 
(there are plenty of wrinkles in this formula, but this is the general 
idea).

The beauty of this approach is that you needn’t dream up “The Big 
Exhaustive List of Words and Phrases That Indicate a Message Is/Is Not 
Spam.” The filter naively calculates a statistical fingerprint for 
spam and not-spam, and checks the new messages against them.

This approach -- and similar ones -- are evolving into an immune 
system for the Internet, and like all immune systems, a little bit 
goes a long way, and too much makes you break out in hives.

ISPs are loading up their network centers with intrusion detection 
systems and tripwires that are supposed to stop attacks before they 
happen. For example, there’s the filter at the hotel I once stayed at 
in Jacksonville, Fla. Five minutes after I logged in, the network 
locked me out again. After an hour on the phone with tech support, it 
transpired that the network had noticed that the videogame I was 
playing systematically polled the other hosts on the network to check 
if they were running servers that I could join and play on. The 
network decided that this was a malicious port-scan and that it had 
better kick me off before I did anything naughty.

It only took five minutes for the software to lock me out, but it took 
well over an hour to find someone in tech support who understood what 
had happened and could reset the router so that I could get back online.

[snip]

NetHappenings Mailing List | Network Newsletters | Science | Technology
Friday, November 23, 2007 8:07:07 PM (Eastern Standard Time, UTC-05:00)    Disclaimer  |  Comments [0]  |  Related posts:
[EC] NetHappenings News and Resources
The Slow Burn
RIP Majel Barrett
China New Year January 26
CURRENT STATUS ON THE PROTECTION AND LEGISLATION OF NATIONAL FOLKLORE IN CHINA
Federal Reserve creates the Term Asset-Backed Securities Loan Facility (TALF)