scaling drupal step two - sticky load balancing with apache mod_proxy

if you've setup your drupal deployment with a separate database and web (drupal) server (see scaling drupal step one - a dedicated data server), a good next step, is to cluster your web servers. drupal generates a considerable load on the web server and can quickly become resource constrained there. having multiple web servers also increases the the redundancy of your deployment. as usual, my examples are for apache2, mysql5 and drupal5 on debian etch. see the scalability overview for related articles.

one way to do this is to use a dedicated web server running apache2 and mod_proxy / mod_proxy_balancer to load balance your drupal servers.

deployment overview

this table summaries the characteristics of this deployment choice

scalability:	fair
redundancy:	fair
ease of setup:	fair

servers

in this example, i use:

web server	drupal-lb1.mydomain.com	192.168.1.24
web server	drupal-lb2.mydomain.com	192.168.1.25
data server	drupal-data-server1.mydomain.com	192.168.1.26
load balancer	apache-balance-1.mydomain.com	192.168.1.34

network diagram

load balancer setup: install and enable apache and proxy_balancer

create a dedicated server for load balancing. install apache2 (apt-get install apache2) and then install mod proxy_balancer and proxy_http with dependencies

# a2enmod proxy_balancer
# a2enmod proxy_http

enable mod_proxy in mods-available/proxy.conf. note that i'm leaving ProxyRequests off since we're only using the ProxyPass and ProxyPassReverse directives. this keeps the server secure from spammers trying to use your proxy to send email.

configure mod_proxy and mod_proxy_balancer

mod_proxy and mod_proxy balancer serve as a very functional load balancer. however mod_proxy_balancer makes slightly unfortunate assumptions about the format of the cookie that you'll use for sticky session handling. one way to work around this is to create your own session cookie (very easy with apache). the examples below describe how to do this

first create a virtual host or use the default (/etc/apache2/sites-available/default) and add this configuration to it:

note:

• i'm allowing access to the balancer manager (the web UI) from any IP matching 192.168.*.*
• i'm load balancing between 2 servers (drupal-lb1.mydomain.com, drupal-lb2.mydomain.com) on port 80
• i'm defining two routes for these servers called lb1 and lb2
• i'm excluding (!) the balancer-manager directory fro the ProxyPass to allow access to the manager ui on the load balancing server
• i'm expecting a cookie called BALANCEID to be available to manage sticky sessions
• this is a simplistic load balancing configuration. apache has many options to control timeouts, server loading, failover etc. too much to cover but read more in the apache documentation

configure the web (drupal) servers to write a session cookie

on each of the web (drupal) servers, add this code to your vhost configuration:

RewriteEngine On
RewriteRule .* - [CO=BALANCEID:balancer.lb1:.mydomain.com]

making sure to specify the correct route e.g. lb1 on drupal-lb1.mydomain.com etc.

you also probably want to setup your cookie domain properly in drupal, i.e. modify drupal/sites/default/settings.php as follows:

# $cookie_domain = 'example.com';
$cookie_domain = 'mydomain.com';

important urls

useful urls for testing are:

• balancer manager ui: http://apache-balance-1.mydomain.com/balancer-manager
• direct access to drupal on lb1: http://drupal-lb1.mydomain.com/drupal/
• direct access to drupal on lb1: http://drupal-lb2.mydomain.com/drupal/
• access to drupal through the load balancer: http://apache-balance-1.mydomain.com/drupal/

the balancer manager

the mod_proxy_balancer ui enables point-and-click update of balancer members.

the balancer manager allows you to dynamically change the balance factor or a particular member, change it's route or put it in the off line mode.

debugging

to debug your configuration it's useful to turn up apache's debugging level on your apache load balancer by adding this to your vhost configuration:

LogLevel debug

this will produce some very useful debugging output (/var/log/apache2/error.log) from the proxying and balancing code.

firefox's cookie viewer tools->options->privicy->show cookies is also useful to view and manipulate your cookies.

if you plan to experiment with bringing servers up and down to test them being added and removed from the cluster you should consider setting the "connection pool worker retry timeout" to a value lower than the default 60s. you could set them to e.g. 10s by changing your configuration to the one below. a 10s timeout allows for quicker test cycles.

BalancerMember http://drupal-lb1.scream.squaretrade.com:80 route=lb1 retry=10
BalancerMember http://drupal-lb2.scream.squaretrade.com:80 route=lb2 retry=10

next steps

one single-point-of-failure in this deployment is the apache load balancer. consider clustering your load balancer with scaling drupal step three - using heartbeat to implement a redundant load balancer

references and documentation

tech blog

if you found this article useful, and you are interested in other articles on linux, drupal, scaling, performance and LAMP applications, consider subscribing to my technical blog.