I'm working with data that has been mined for clickers/openers, and it's been run through Impressionwise about a year ago, but I still had Webjuice scrub out over 50% of clicks as apparent bot traffic. So, I've started trying to track bot clicks by putting a tracking link labeled as "click here" in an html commented out section so that I can gather the offending email addresses to create a scrub list from. Is this this best way to reduce bot traffic? Or does anyone have a better way to do this?
Bot traffic is not based off the email addresses. Whoever telling you that they remove bots by removing email addresses is lying to you unfortunately. I have bots clicking on my personal gmail/aol/yahoo email accounts. Anyone with a client based anti-spam applyance would be seen as bot activity since most of them go through the links to check the landing page. Bot clicking must be handled at the email platform level of things, specifically at your redirector. Your redirector must be smart enough to detect if a "click" is coming from a known bot IP and redirect it properly (and not to block it). There are threads here that would give you the entire list of known bot IPs for you to check. What you do with that traffic is strictly up to you. I hope it helps...
You won't be able to remove bots.. Like DAgent said, there's countless software and hardware packages that scan links in email and send them off to be checked.. Your best bet would be to handle bot detection with your link tracking/redirection software.. You can do some simple checks on the user agent, IP address, geo location, or even javascript to weed out the bots and redirect them.
Thanks for the heads up on this. I've been doing work for a jewelry company and running into similar issues.
These appear to be lists to scrub out search engine bots/spiders? Or am I not seeing the spam bot list(s)?
I bought the maxmind geoip city and org database. I took all of my opens/clicks from the last few months and did lookups on the IPs, storing the results in a database. I then grouped the IPs by org and amount of actions. I was able to find quite a few commercial org ranges that consistently hit my links .. I used that data to build up a nice list of IPs to redirect to some static page.
Yep, mostly search engine bots, but there are some non-search bots under the "Non Spiders" link (http://www.iplists.com/non_engines.txt). We were getting a ton from Cyveillance. But this is by no means an exhaustive list, just a starting point.
we found that detecting if a cookie drop is working within your redirector, filters out efficiently the bots
Place a pixel on your landing page. Robots wont display images while crawling you landing page (if they do -- 95% of them wont). Every click tracked in your system that never reached the landing page, is a bot.
