APEWS.ORG data usage

Due to the decline in use / effectiveness of Usenet for antispam, which formerly had newsgroups called news.admin.net-abuse.blocklisting and news.admin.net-abuse.sightings (both now inactive), here is a place where users of APEWS.ORG database can publish their experience and any problems or errors. On their website http://www.APEWS.ORG have asked for evidence of errors so that they can improve their data, and to my knowledge there is no one place that exists for the purpose.

In choosing an antispam solution for email servers, there are point scoring methods like SpamAssassin and there are Domain Name Server Block List (DNSBL). Operating my own business servers and not having the resources of perhaps larger corporations, I needed an approach that would seriously cut into the, at times, overwhelming number of inbound emails. I came across Apews.org shorly after SPEWS ceased to be updated and after finding other DNSBLs to be inadequate. However, Google for this DNSBL and the majority of information available suggests that the data represents an overly aggressive approach with too many false positives.

Rather than just believe what I found, especially since there was little evidence of the false positives, I decided to test the IP based L2.APEWS.ORG data which they recommend for scoring to be used in addition to other DNSBL services like Spamhaus, Spamcop, Sorbs etc via SpamAssassin or similar. Knowing your clients and having a good white list is essential these days and I doubt anyone would put an email server online without one. My own whitelist has come from more than 10 years of experience and accumulated knowledge thus testing inbound connections first against the whitelist, before the blacklist, produces almost zero errors.

During the last few years there have been very few statistics websites that compared True [TP] and False [FP] Positives from the use of a single DNSBL e.g. Spamhaus, Spamcop, UCEProtect or Sorbs etc. By 2008 it seemed that L2.APEWS.ORG had a very high [90%] spam catch rate [TP] together with a reducing level of false positives [FP] but the website operator didn't elaborate much on the FP, only referring to the mail stream as being USA based and including some marketing emails for products, services, and reviews of same. No other free DNSBL comes close from what I can see, http://www.UCEProtect.net have 3 databases and if all are used then their results appear to be about the same if not a little better, approx 1% to 1.5% higher TP currently.

I have configured my email servers to use the L2.APEWS.ORG DNSBL in realtime during the SMTP session, after first querying the whitelist. In the event that the IP address is unknown or untrusted and then found to be listed in L2.APEWS.ORG then the email servers don't reject the email, they just flag it as probable spam by the use of an X-Header which a script then uses to move the flagged emails to the user's spam / junk folder.

Note my criteria and requirements;

• Email servers are used for both local and remote users
• Users to/from UK, West/Central Europe, USA, India, Australia, also roaming users including Far East

Source of emails [almost all private emails are via free webmail providers]

• approx 40% are received are via Yahoo, Gmail, AOL & Hotmail users

• approx 40% are received via contracted email services [negligible spam received] e.g. messagelabs, psmtp, frontbridge, bigfish, postini, mxlogic etc

• approx 15% are from client owned corporate servers [negligible spam received] and includes many international and regional banks, USA / EU government departments etc

• approx 5% are newsletters and social networking contact [negligible spam received] e.g. reuters, alertnet, foxnews, cnn, nytimes, dartmail, collab, cheetah, ezinedirector, sun microsystems, symantec, linkedin, facebook, myspace, flickr, digg, naymz.com [chnaged back after being visible.me], mbox, j2global, iht, osac.gov, oecd, imf.org, worldbank, natgeo, dhl, ups, fedex, deutscheposte, usps, dealertime, shopping.com, amazon.com, aa.com, continental air, virgin, travelocity, hotel.com, cheaptickets, lufthansa etc
  

• hard/soft-ware suppliers & manufacturers e.g. HP, Dell, Cisco, Microsoft, Apple, Macromedia, Adobe, sourceforge etc

• Very few emails are received via ISP smtp servers / smart hosts [negligible spam received] e.g. rogers, rogerstelcom, earthlink, mindspring, prodigy, comcast, sprint, sprintlink, btinternet, bt.com, demon, shaw, shawcable, qwest, adelphia, bellatlantic, bell, bellglobal, bellsouth, bellnexxia, swbell, bellhosting, att, ownmail, telstra, megacity, free2surf, charter, level3, optus, sonic, orange, vodafone, pipex, t-online, dtag, t-mobil, cox, coxinternet, verizon, cogentco, blueyonder, bigpond, roadrunner, twtelecom, nortel etc
  

• Almost zero emails are received via domain Registrars [negligible spam received] e.g. networksolutions, netsol, register, joker, gandi, godaddy, tucows etc
  

• Complaints by email relating to abuse from my servers can be received to role accounts here from major dnsbl operators for each domain name hosted e.g. spamhaus, spamcop, sorbs, abuseat, ahbl, uceprotect, robtex, njabl, mail-abuse, uceb, abuse-net, whitelisted trusted sender servers

• all the above are regarded as trusted senders and as such have been whitelisted here

• the only spam received into user inboxes comes almost entirely from free webmail user accounts or unlisted IP addresses, True Positive is better than 99% because of a good whitelist
  

• all emails in a user's spam/junk folder have been found to be spam i.e. correctly identified and after running email client spam filters on the mailboxes. The FP% is extremely low, less than 0.05%.
  

In conclusion, the use of L2.APEWS.ORG has, for us, removed the spam problem to the extent that the few spam we do receive are via Yahoo, Gmail, AOL and Hotmail servers that we need to give access. It has been said that of the world's total daily email volume, approx 97% is unsolicited bulk email and our experience accords with that statistic. The remaining 3% of the world's total daily email volume is solicited and the above figures represent an approximate analysis of the source and/or nature as it pertains to our business mail stream. These are our findings and no warranty either express or implied exists regarding these findings since each mail stream is unique to the particular business or network.