[ILUG] Rate limiting apache requests per second

[Ronan Mullally]

I've got a site with a couple of muppets that think they can DOS (not
DDOS) it.  I've got numerous iptables bells and whistles in place to
mitigate against this, the most effective being connlimit statements that
limit the number of concurrent connections from a particular source IP
address to the site.  This limits the amount of disruption caused by any
particular source, but can result in substantial bandwidth spikes as the
limited number of connections that are allowed hammer away on the site.

I'm trying to limit the number of requests per second from a given source
IP to address this.  Bonus points for being able to limit the number of
connections per second from a given source IP to a particular URI.

Before you shout "mod_cband" or "mod_qos" or "mod_evasive" I've got the
following setup:


             Muppet
               |
          [ Internet ]
               |
               |
         Reverse Proxy
            (pound)
               |
        +------+------+
        |      |      |
          Web Servers
           (apache)


Since everything is getting reverse proxied the standard apache rate
limiting solutions won't work as everything appears to come from the same
source (the reverse proxy).  I've not found a module that will allow me to
use the X-Forwarded-For header my reverse proxy adds in place of the
source IP address as a key for limiting.  Replacing the reverse proxy with
something transparent (like a LVS instance) isn't an option at this stage.

The reverse proxy would be the obvious place to apply request limiting,
but pound doesn't have this facility.

An iptables rule that uses the string and hashlimit/recent modules to
restrict the number of packets with ^GET  is one very ugly option (I've
just thought of but not tried yet).  The only other alternative I can
think of would be to have a daemon parsing syslog output that triggers the
addition and removal of iptables rules to block mis-behaving source
addresses (ie those that excessively trigger the existing connlimit rule),
however that would not be as clean as I'd like.

Does anybody have any other suggestions?


-Ronan