Moving wordpress from a subdirectory – and generation of 301 redirects from a google sitemap
So I mistakenly have installed wordpress in a subdirectory on my site - this worked well for testing, but I didn't buy this domain to have my entire site sitting within a subdirectory!
Wordpress has a good guide on this subject, which has step by step instructions on migrating your installation from one directory to another (including the root directory of your server):
http://codex.wordpress.org/Moving_WordPress
The only problem is that once you do this, if you have indexed content in google (or links from other blogs) - you will now miss out on those links getting through to your site. At this point you need to set up some redirects to your new site, which, if you have a large site, could be a real hassle if you're manually adding 301 redirects to a .htaccess file. Another way to do this is to use a pre-existing google sitemap file (sitemap.xml). In this case I used a newly created one at the sites' new location, outside of the 'eric' directory. There was no difference in site links between moving the site outside of the 'eric' directory, so I can use this .xml without removing any nonexistent links.
I used a single line of not-so-elegant awk code from the bash command line to generate the required 301 redirects, so that any old links will now link to my non 'eric' directory links:
cat ../sitemap.xml | grep thern.org | awk -F"<loc>" '{print $2}' | awk -F"</loc>" '{print $1}' | awk -F"http://thern.org" '{print "Redirect 301 /eric"$2" http://thern.org"$2}'
Obviously if you're running this on your own system, you'll have to edit 'thern.org' to be your domain, '/eric' to be your current 'old' directory, and "../sitemap.xml" as the location of your sitemap. If you don't have console access or don't know awk, this probably doesn't help you!
Here's a snippet of the .xml file:
<loc>http://thern.org/</loc>
<lastmod>2010-02-24T21:00:42+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>http://thern.org/linux-and-freebsd/yum-update-check-script-runs-via-crontab-and-emails-when-new-updates-are-available/</loc>
<lastmod>2010-02-23T19:51:35+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.2</priority>
</url>
<url>
<loc>http://thern.org/recent-posts/</loc>
<lastmod>2010-02-22T07:20:14+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.7</priority>
</url>
<url>
And here's the resulting output:
Redirect 301 /eric http://thern.org
Redirect 301 /eric/ http://thern.org/
Redirect 301 /eric/linux-and-freebsd/yum-update-check-script-runs-via-crontab-and-emails-when-new-updates-are-available/ http://thern.org/linux-and-freebsd/yum-update-check-script-runs-via-crontab-and-emails-when-new-updates-are-available/
Redirect 301 /eric/recent-posts/ http://thern.org/recent-posts/
Paste this into a .htaccess file in the /eric/ directory, and presto, all the links now end up at the right place - google will be informed of this result, and will update their database accordingly, and anyone who used these links to link into this site will now go to the right place rather than a 404 or another error.
FreeBSD Plesk 8 – installing php5 as CGI alongside php4
If you have ever needed php5 to run along side php4, you can always compile php5 as a CGI, and let php4 remain as an apache module.
Here we've compiled php5 from FreeBSD's ports, and have installed it as a CGI so that specific vhosts will run on this new php5. The installation was done on a Plesk 8 box running FreeBSD 6.x.
Compiling PHP5 (installation from FreeBSD Ports)
1) Update port tree using portsnap (use `portsnap extract` instead of update if this is the first time you've used portsnap). Or use classic rsync techniques.
portsnap fetch; portsnap update
2) Configure php5 and install:
cd /usr/ports/lang/php5
make config; make install
In the config, the basics should already be selected, make sure you have not selected php5 as an apache module, and keep debugging turned off unless you require this.
3) Compile the php5 extensions.
cd /usr/ports/lang/php5-extensions
make config; make install
In the config, make sure you have selected the extensions that you require. For my purposes, I've selected: BZ2, CTYPE, CURL, DOM, EXIF, FILTER, GD, GETTEXT, HASH, ICONV, IMAP, JSON, MCRYPT, MHASH,MYSQL, NCURSES, PCRE, PDO, PDO_SQLITE, POSIX, SESSION, SIMPLEXML, SOAP, SPL, SQLITE, TOKENIZER, XML, XMLREADER, XMLRPC, XMLWRITER, XSL, ZLIB.
4) Create the directory for your installation of php, and make sure it is owned by the proper user (the siteowner) so that suexec will be happy running binaries in this directory:
mkdir /usr/local/psa/home/vhosts/yoursite.com/httpdocs/php5-cgi
chown siteowner:psacln /usr/local/psa/home/vhosts/yoursite.com/httpdocs/php5-cgi
5) Copy the new php-cgi over to this directory (we're naming the php-cgi binary 'php') and make sure to set the ownership to the siteowner, so that suexec can run this binary.
cp /usr/local/bin/php-cgi /usr/local/psa/home/vhosts/yoursite.com/httpdocs/php5-cgi/php
chown siteowner:psacln /usr/local/psa/home/vhosts/yoursite.com/httpdocs/php5-cgi/php
6) Move the recommended configuration file to the real configuration file location, edit if necessary:
mv /usr/local/etc/php.ini-recommended /usr/local/etc/php.ini
Now we add the vhost configuration and apache bits:
Add the following to your /usr/local/psa/home/vhosts/yoursite.com/conf/vhost.conf (change "yoursite.com" to your domain name or host)
ScriptAlias /php5-cgi /usr/local/psa/home/vhosts/yoursite.com/httpdocs/php5-cgi
Action application/x-httpd-php5-custom "/php5-cgi/php"
AddType application/x-httpd-php5-custom .php
AddType application/x-httpd-php5-custom .php5
Run the reconfiguration utility and restart apache:
/usr/local/psa/admin/sbin/my_apci_rst
/usr/local/psa/rc.d/httpd restart
Test out the page with a phpinfo.php script, you can create one easily from the command line, go into the httpdocs directory and run this:
echo "<?php
phpinfo();
?> " > phpinfo.php
From a browser, go to yoursite.com/phpinfo.php and see if you have the new php you just installed showing up. If you do, you have succeeded. If not, or if there are any errors, here are a few things to try:
Troubleshooting
Check the local site error_log:
/usr/local/psa/home/vhosts/yoursite.com/statistics/logs/error_log
Check the suexec log:
/usr/local/psa/apache/logs/suexec_log
Try running php from the command line:
/usr/local/bin/php -v
If you see any errors on the command line, this will result in the binary not working from Apache.
One error I ran into was this:
/usr/local/bin/php --version
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20060613/xsl.so' - Shared object "libgpg-error.so.3" not found, required by "libexslt.so.8" in Unknown on line 0
PHP 5.2.5 with Suhosin-Patch 0.9.6.2 (cli) (built: Nov 26 2007 03:33:26)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
This was caused by some ports being older versions. To solve this, you must make sure that the libgpg-error and related ports are up to date, and then recompile the php5-xsl port:
recompile (from packages -P) libgpg-error and related ports:
portupgrade -rf libgpg-error-\* -P
Recompile xsl related ports, I may be recompiling too many, but this worked for me:
cd /usr/ports/textproc/libxslt
make clean; make deinstall; make install
cd /usr/ports/textproc/php5-xsl
make clean; make deinstall; make install
cd /usr/ports/textproc/php5-xml
make clean; make deinstall; make install
This solved the shared library problem, and allowed php5 to work. You shouldn't have to recompile php after this, since this only impacts the shared libraries.
Double check your /usr/local/etc/php/extensions.ini to make sure that you have all the libraries installed that you require.
Dealing with Apache referrer spam
I've had a number of sites get hammered by referrer spam lately, and had to look into methods of stopping, or at least slowing the attacks.
Referrer spam is when a group of random (probably trojan or hacked computers) or single computers hit random pages on your site with faked referrer values, typically all pointing to different subpages of a single site. This happens within a few minutes or less, and can also cause a ddos attack if there are too many requests within that time and if Apache cannot deal with that many requests.
In my experience, I've found these requests happen in bursts of 60 - 120+ (can differ quite a bit though).
This causes many problems:
1) Apache has to spawn processes to deal with the 100+ requests from different IP's
2) Memory usage can go through the roof if each of these requests requires a lot of other files (ie. if they request gallery pages which have a number of large images)
3) Once your server starts swapping, everything else slows down and ...
4) This can result in all these requests taking longer to respond, which can result in all your swap being used, which can cause a ddos while the disk I/O is through the roof.
Implementing good Apache limits, having enough ram, enabling caching programs such as phpa, xcache and similar will help you through this sort of situation. However, the best plan is to block or limit the severity of how many of these requests can get through at any given time.
The first step is to determine if you've got a referrer spam problem.
In order to detect this referrer spam, you can run the following (or similar) command on your Apache logs:
cat access_log | awk '{print $11}' | egrep -v '(your-website.com|google|yahoo|msn.com)' | awk -F"/" '{print $3}' | sort | uniq -c | sort -k 1
118 www.chargecardonlinedepo.com
176 www.365creditcard.com
190 www.setcreditcard.com
This code assumes that the 'referrer' section of the log is an awk key of 11. We want to remove all mention of your own website and different searching sites: egrep -v '(your-website.com|google|yahoo|msn.com)' -- which takes out your-website.com, google, yahoo, and msn from the referrers that are returned by this command. This is not a complete list, however, so feel free to edit.
The code returns the number of times the specific referrer has accessed the site, and the referrer base address (rather than the full address, which is usually random)
An example from the apache log is as follows:
...
218.58.136.4 - - [28/Sep/2007:12:42:16 -0400] "GET /modules.php?name=Gallery&file=details&image_id=16226&sessionid=a84f2b586aae1fc6173168704a274a61 HTTP/1.1" 403 291 "http://www.365creditcard.com/plastic-cards-online9.html" "Mozilla/4.0 (compatible; MSIE 5.0; Windows ME) Opera 5.11 [en]"
189.19.210.206 - - [28/Sep/2007:12:42:17 -0400] "GET /modules.php?name=Gallery&file=details&image_id=18059&sessionid=507a7a9c194a72002b0f9c2d4e84d2c0 HTTP/1.1" 403 291 "http://www.365creditcard.com/social-responsible-business-credit-card.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; iOpus-I-M)"
...
Keep in mind that in that grep -e statement, you might miss some referrer spam if the spammers decide to include parts of 'google' or 'yahoo' in their links. Such as "http://www.365creditcard.com/social-responsible-business-credit-google.html" or similar. I doubt they would, but hey, you never know.
The Apache log snippet from above shows these requests resulting in a 403 error response from the server. This is because I've got a mod_rewrite rule in place that blocks these sorts of requests on the server.
Blocking these sorts of referrer requests with mod_rewrite
The following mod_Rewrite rule can be placed within your .htaccess or in Apache's httpd.conf file (or your vhost file). This blocks a lot of types of referrer spam. The original list was obtained from http://www.joemaller.com/htaccess.txt , this one is edited quite a bit, and a lot of referrer sites added.
Rules last updated: Oct 10, 2007
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?adult(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?anal(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?blow.?job(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?gay(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?cum+shot(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?casino(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?incest(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?mature(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?nude(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?tits(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?titten(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?wichsab(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?wichslos(-|.).*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*affiliplanet.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*apart-?design.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*auktion.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*autogewinne24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*autospiele24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*babay.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*euromillionen.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*eurowins.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*geldspiele24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*goovle.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*gsm-support.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*gzltax.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*heil-fasten.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*immobiliengewinne24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*internetsupervision.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*keywordmaster.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*nackt-stars-nackt.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*one2onemag.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*qw8.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*referrer-script.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*ranking-hits.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*reisegewinne24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*rootfood.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*shemale.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*single66.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*slamhost.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*spielepsychatrie.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*superface.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*texasholdem.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*topgewinn24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*topspiele24.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*transexual.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*usa-wins.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*vendini.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*webmasterplan.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*wichsfick.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*wseeker.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*yachtdurak.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*xmaster.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://www14\.blogspot.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*insur.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*chargecard.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*creditcard.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*automobile.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*finance.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*autos.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*pharm.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*motors.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*gambl.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*playlisT.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*gratis.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*yahh+oo.*$
RewriteRule .* - [F,L]
Blocking access before or after the attack / spam?
Pre-Emptive
The above rewrite rules are pre-emptive spam stopping, that is, they will stop referrer spammers who match the rules before it happens. Of course if the spammers try to spam with totally random URL's that make no sense, then it's all over for this method, however, part of the reason to use referrer spam is that you've got tons of your brand-name-URL's at the top of some list.
Reactive
After this has happened for a while, you can also block based on IP address from your logs. I've found this helps, since a number of the hosts seem to recur quite often while spamming. Even though the referrer spam method uses semi-random IP's, if you've got a lot of logs, you can find out that the same IP's are coming back again and again.
Blocking offending IP addresses with mod_rewrite
I've narrowed down a number of hosts (based on their IP address) that are quite promiscuous with this type of referrer spam on my server. I'm sure this is only a set of hosts that may be trojan or drone home computers that might be patched at any moment now (and then should be taken off this list), but blocking these hosts has helped a great deal.
To gather evidence from logs, output to a rewrite rule, and sort based on IP:
cat access_log | zcat access_log*gz | awk '{print $1 "\t" $11}' | egrep -e casino -e horny -e bargain -e craps -e hentai -e office -e pill -e viagra -e poker -e klingel -e credit -e cialis -e insur -e texas -e holdem -e pharm -e gambl -e gratis -e charge -e automobile -e finance -e autos | awk '{print $1}' | sort | uniq -c | sort -k 1 -n | awk '{ if ( $1 > 19 ) print $1 "\t" $2}' | awk '{print "RewriteCond %{REMOTE_ADDR} ^"$2"$ [OR]"}' | sort -k 2 -n -t"^"
This code outputs hosts that match the specific egrep statement here and have requested pages more than 19 times. I've scanned all the old logs + new logs, hence the zcat access_log*gz . Edit the egrep statement to suit your needs. If your site deals with anything relating to what we're grepping here, or you get 'real' traffic from referrers with these sorts of words in their URL's, then you probably want to double check this before you block anything!
Here's the output of this command, plus a RewriteEngine On at the top for inclusion into .htaccess (if you include this in your httpd.conf or vhost.conf files, you'll need only one RewriteEngine On statement). In fact, you might need just one nomatter which area it is in, in .htaccess or your main config, I haven't actually double checked this.
RewriteEngine On
RewriteCond %{REMOTE_ADDR} ^58.247.2.108$ [OR]
RewriteCond %{REMOTE_ADDR} ^58.71.35.206$ [OR]
RewriteCond %{REMOTE_ADDR} ^59.41.39.21$ [OR]
RewriteCond %{REMOTE_ADDR} ^59.42.250.145$ [OR]
RewriteCond %{REMOTE_ADDR} ^59.77.16.170$ [OR]
RewriteCond %{REMOTE_ADDR} ^60.190.223.77$ [OR]
RewriteCond %{REMOTE_ADDR} ^60.190.249.66$ [OR]
RewriteCond %{REMOTE_ADDR} ^61.128.110.110$ [OR]
RewriteCond %{REMOTE_ADDR} ^61.142.81.37$ [OR]
RewriteCond %{REMOTE_ADDR} ^61.152.248.210$ [OR]
RewriteCond %{REMOTE_ADDR} ^61.156.42.123$ [OR]
RewriteCond %{REMOTE_ADDR} ^61.175.139.6$ [OR]
RewriteCond %{REMOTE_ADDR} ^61.253.10.18$ [OR]
RewriteCond %{REMOTE_ADDR} ^62.215.3.45$ [OR]
RewriteCond %{REMOTE_ADDR} ^64.242.24.13$ [OR]
RewriteCond %{REMOTE_ADDR} ^65.23.153.132$ [OR]
RewriteCond %{REMOTE_ADDR} ^66.167.100.59$ [OR]
RewriteCond %{REMOTE_ADDR} ^69.77.169.2$ [OR]
RewriteCond %{REMOTE_ADDR} ^69.77.171.34$ [OR]
RewriteCond %{REMOTE_ADDR} ^78.147.105.184$ [OR]
RewriteCond %{REMOTE_ADDR} ^80.78.23.226$ [OR]
RewriteCond %{REMOTE_ADDR} ^82.193.10.182$ [OR]
RewriteCond %{REMOTE_ADDR} ^84.22.67.115$ [OR]
RewriteCond %{REMOTE_ADDR} ^84.87.124.40$ [OR]
RewriteCond %{REMOTE_ADDR} ^85.136.165.139$ [OR]
RewriteCond %{REMOTE_ADDR} ^85.25.140.160$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.101.209.10$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.101.244.6$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.101.244.7$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.101.244.8$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.101.244.9$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.106.133.217$ [OR]
RewriteCond %{REMOTE_ADDR} ^87.248.160.134$ [OR]
RewriteCond %{REMOTE_ADDR} ^88.48.65.148$ [OR]
RewriteCond %{REMOTE_ADDR} ^89.202.193.225$ [OR]
RewriteCond %{REMOTE_ADDR} ^89.97.0.9$ [OR]
RewriteCond %{REMOTE_ADDR} ^91.121.67.145$ [OR]
RewriteCond %{REMOTE_ADDR} ^124.128.14.121$ [OR]
RewriteCond %{REMOTE_ADDR} ^124.254.63.158$ [OR]
RewriteCond %{REMOTE_ADDR} ^125.46.36.223$ [OR]
RewriteCond %{REMOTE_ADDR} ^140.119.98.31$ [OR]
RewriteCond %{REMOTE_ADDR} ^140.247.53.18$ [OR]
RewriteCond %{REMOTE_ADDR} ^146.164.34.14$ [OR]
RewriteCond %{REMOTE_ADDR} ^152.31.229.186$ [OR]
RewriteCond %{REMOTE_ADDR} ^189.21.156.2$ [OR]
RewriteCond %{REMOTE_ADDR} ^190.129.118.1$ [OR]
RewriteCond %{REMOTE_ADDR} ^190.39.173.149$ [OR]
RewriteCond %{REMOTE_ADDR} ^193.111.198.56$ [OR]
RewriteCond %{REMOTE_ADDR} ^193.194.69.155$ [OR]
RewriteCond %{REMOTE_ADDR} ^193.194.69.157$ [OR]
RewriteCond %{REMOTE_ADDR} ^193.202.63.138$ [OR]
RewriteCond %{REMOTE_ADDR} ^193.52.195.6$ [OR]
RewriteCond %{REMOTE_ADDR} ^194.112.113.78$ [OR]
RewriteCond %{REMOTE_ADDR} ^194.9.85.141$ [OR]
RewriteCond %{REMOTE_ADDR} ^195.190.158.2$ [OR]
RewriteCond %{REMOTE_ADDR} ^195.246.155.219$ [OR]
RewriteCond %{REMOTE_ADDR} ^195.76.242.227$ [OR]
RewriteCond %{REMOTE_ADDR} ^196.34.133.63$ [OR]
RewriteCond %{REMOTE_ADDR} ^199.243.251.243$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.199.220.110$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.31.42.3$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.32.253.146$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.42.228.234$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.83.4.3$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.83.4.4$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.83.4.5$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.83.4.6$ [OR]
RewriteCond %{REMOTE_ADDR} ^200.87.117.58$ [OR]
RewriteCond %{REMOTE_ADDR} ^201.133.184.168$ [OR]
RewriteCond %{REMOTE_ADDR} ^201.251.64.94$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.101.107.120$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.105.182.87$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.141.148.18$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.145.127.18$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.155.219.88$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.170.51.11$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.194.202.7$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.64.220.99$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.74.244.13$ [OR]
RewriteCond %{REMOTE_ADDR} ^202.96.189.45$ [OR]
RewriteCond %{REMOTE_ADDR} ^203.128.4.39$ [OR]
RewriteCond %{REMOTE_ADDR} ^203.160.1.39$ [OR]
RewriteCond %{REMOTE_ADDR} ^203.177.180.38$ [OR]
RewriteCond %{REMOTE_ADDR} ^203.186.92.119$ [OR]
RewriteCond %{REMOTE_ADDR} ^203.69.39.251$ [OR]
RewriteCond %{REMOTE_ADDR} ^206.51.229.7$ [OR]
RewriteCond %{REMOTE_ADDR} ^207.250.10.170$ [OR]
RewriteCond %{REMOTE_ADDR} ^207.44.238.95$ [OR]
RewriteCond %{REMOTE_ADDR} ^207.59.107.175$ [OR]
RewriteCond %{REMOTE_ADDR} ^207.67.117.173$ [OR]
RewriteCond %{REMOTE_ADDR} ^209.128.75.180$ [OR]
RewriteCond %{REMOTE_ADDR} ^209.59.121.122$ [OR]
RewriteCond %{REMOTE_ADDR} ^209.99.227.70$ [OR]
RewriteCond %{REMOTE_ADDR} ^210.16.47.7$ [OR]
RewriteCond %{REMOTE_ADDR} ^210.192.111.173$ [OR]
RewriteCond %{REMOTE_ADDR} ^210.34.14.186$ [OR]
RewriteCond %{REMOTE_ADDR} ^210.42.140.5$ [OR]
RewriteCond %{REMOTE_ADDR} ^210.51.51.24$ [OR]
RewriteCond %{REMOTE_ADDR} ^211.136.105.86$ [OR]
RewriteCond %{REMOTE_ADDR} ^211.14.6.216$ [OR]
RewriteCond %{REMOTE_ADDR} ^211.140.138.39$ [OR]
RewriteCond %{REMOTE_ADDR} ^211.154.104.85$ [OR]
RewriteCond %{REMOTE_ADDR} ^212.241.180.42$ [OR]
RewriteCond %{REMOTE_ADDR} ^212.72.30.140$ [OR]
RewriteCond %{REMOTE_ADDR} ^213.132.44.92$ [OR]
RewriteCond %{REMOTE_ADDR} ^213.170.35.27$ [OR]
RewriteCond %{REMOTE_ADDR} ^213.239.218.131$ [OR]
RewriteCond %{REMOTE_ADDR} ^216.84.23.206$ [OR]
RewriteCond %{REMOTE_ADDR} ^217.174.98.198$ [OR]
RewriteCond %{REMOTE_ADDR} ^217.208.126.186$ [OR]
RewriteCond %{REMOTE_ADDR} ^217.237.173.35$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.104.180.228$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.22.112.166$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.25.214.216$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.28.207.44$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.28.46.250$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.56.60.11$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.58.136.4$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.7.13.186$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.75.68.205$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.80.215.231$ [OR]
RewriteCond %{REMOTE_ADDR} ^218.97.217.34$ [OR]
RewriteCond %{REMOTE_ADDR} ^219.138.204.162$ [OR]
RewriteCond %{REMOTE_ADDR} ^219.209.194.156$ [OR]
RewriteCond %{REMOTE_ADDR} ^219.240.36.173$ [OR]
RewriteCond %{REMOTE_ADDR} ^219.240.36.175$ [OR]
RewriteCond %{REMOTE_ADDR} ^219.83.66.205$ [OR]
RewriteCond %{REMOTE_ADDR} ^220.169.248.21$ [OR]
RewriteCond %{REMOTE_ADDR} ^220.194.59.252$ [OR]
RewriteCond %{REMOTE_ADDR} ^220.247.234.141$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.12.134.132$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.12.56.91$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.202.118.17$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.226.224.2$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.232.159.112$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.233.134.87$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.239.90.209$ [OR]
RewriteCond %{REMOTE_ADDR} ^221.4.151.150$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.197.173.53$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.221.6.144$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.236.44.14$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.240.131.194$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.240.208.14$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.66.48.253$ [OR]
RewriteCond %{REMOTE_ADDR} ^222.86.132.20$
RewriteRule .* - [F,L]
Leave the last [OR] off on the last rule, and add RewriteRule .* - [F,L] at the end - this will finish off the set of rules.
So there you have it. This minimizes the impact of referrer spam a great deal by blocking hosts and specific referrer content with mod_rewrite. You'll have to keep up to date with the rules as the spammers evolve with their attack methods.
All rules here tested with Apache 2.0
For further reading:
http://eol.init1.nl/content/view/64/2/
http://www.joemaller.com/refererspam.shtml
http://www.spywareinfo.com/articles/referer_spam/