Network management Toolkit

How much is junk traffic costing you?

Jonathan Yarden Tech Republic

Published: 08 Dec 2005 08:25 GMT

A few weeks ago, a coworker asked me a simple question: How much of the Internet traffic coming into our network was "junk", and how much was this unwanted traffic costing us? Before delving too deeply into his request, I asked him to define the term junk. His classification included suspected port scans, attempts to exploit known weaknesses in applications, and attempted connections to TCP and UDP services on hosts that didn't provide those services.

He asked me to generate a list of offending networks that were the source of junk traffic in the past 30 days. At first, it seemed almost too easy. However, after only a few hours of work, I realised I had underestimated how involved a task it really was.

Finally, after a few days of work, I managed to produce a rather comprehensive list of IP addresses that were sources of junk data. I used a variety of means to gather this data, including NetFlow data, system log files, Snort, and a darknet.

In all, approximately 2.8 million distinct IP addresses from all over the world were responsible for junk traffic on my organisation's network in the past month. And keep in mind that this doesn't include delivered junk email.

Next, I needed to somehow organise these different IP addresses into networks and identify where all the junk was coming from. And this isn't exactly a simple task when you're dealing with so much data.

Since my first step was to aggregate the data, I decided to get a list of the delegated Internet networks from the FTP site of the American Registry for Internet Numbers (ARIN). However, ARIN uses the Border Gateway Protocol (BGP), and the smallest network I could focus on was a /24 or Class C network because of how BGP works.

An hour or two of coding and testing later, and I had an aggregation tool that ordered the junk-sending IP addresses into worldwide networks. Of the approximate 250,000 network paths obtained from ARIN and the 2.8 million junk-sending IP addresses, I had a list of roughly 40,000 networks that were responsible for junk traffic on my organisation's network in the past month.

Next, I used another program to separate the collected data by country into ARIN (North America, the Caribbean, and Southern Africa), APNIC (Asia and the Pacific region), LACNIC (Latin America and...