I noticed a lot of extra "MSIE 6.0" agents in my web server log. I had a quick look around and found it was due to the new version of AVG, which by default scans all links in web search results. This amounted to about 10% of my traffic which I estimated using the following script. This would probably be towards the low end as my site is very linux specific and so probably wouldn't show up on many AVG user's searches.

for i in $(seq 21); do
  ./get_last_days $i access_log |
  grep -E "MSIE 6.0; Windows NT 5.1;( SV1|1813)\)" |
  grep -F '"-"' |
  wc -l
done

So to combat this I've just put the following in my .htaccess file to redirect these requests from AVG linkscanner back to the AVG web site :) Note this should allow valid IE 6.0 users through as they will have a non blank referrer and some other parameters in the user agent string like ".NET" and so will not match the rules below. Of course if I can match it, so can the Malware sites, ho hum.

#Here we assume certain MSIE 6.0 agents are from linkscanner
#redirect these requests back to avg in the hope they'll see their silliness
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1; SV1.$" [OR]
RewriteCond %{HTTP_USER_AGENT} ".*MSIE 6.0; Windows NT 5.1;1813.$"
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{HTTP:Accept-Encoding} ^$
RewriteRule ^.* http://www.avg.com/?LinkScannerSucks [R=307,L]

Hopefully more people will do this, and so show AVG the error or their ways more quickly.

[Update 7th July 2008: AVG have issued a release to say they'll stop scanning search results in the new update being pushed out on the 9th of July. Hopefully not too many stale clients will linger. Result!]