Setting up Drupal on DotCloud's server automation platform

Managing a properly configured server stack is one of the pain points in developing small client sites. Shared hosting is usually sub-par, setting up a VPS from scratch is overkill, and automation/simplification of the server configuration and deployment is always welcome. So I've been very interested in seeing how DotCloud might work for Drupal sites.

DotCloud is in the same space as Heroku, an automated server/deployment platform for Rails applications. DotCloud is trying to cater to much more than Rails, however: they currently support PHP (for websites and "worker" daemons), Rails, Ruby, Python, MySql, Postgresql, and have Node.js, MongoDB and a whole bunch of other components on their roadmap.

The basic idea is to automate the creation of pre-configured EC2 instances using a shell-based API. So you create a web and database setup and push your code with four commands:

dotcloud create mysite
dotcloud deploy -t php mysite.www
dotcloud deploy -t mysql mysite.db
dotcloud push mysite.www ~/code

Each "deployment" is its own EC2 server instance, and you can SSH into each (but without root). The "push" command for deployment can use a Git repository, and files are deployed to the server Capistrano-style, with symlinked releases and rollbacks. (This feature alone, of pushing your code and having it automatically deploy, is invaluable.)

Getting it to work with Drupal is a little tricky. First of all, the PHP instances run on nginx, not Apache. So the usual .htaccess file in core doesn't apply. Drupal can be deployed on nginx with some contortions, and there is a drupal-for-nginx project on Github. However, I write this post after putting in several hours trying to adapt that code to work, and failing. (I've never used nginx before, which is probably the main problem.) I'll update it if I figure it out, but in the meantime, this is only a partial success.

The basic process is this:

• Set up an account (currently needs a beta invitation which you can request)
• Install the dotcloud client using python's easy_install app
• Set up a web (nginx) instance with dotcloud deploy
• Set up a database (mysql or postgres) instance
• Set up a local Git repo, download Drupal, and configure settings.php (as shown with dotcloud info
• Push the repository using dotcloud push
• Navigate to your web instance's URL and install Drupal.
• To use your own domain, set up a CNAME record and run dotcloud alias. ("Naked" domains, i.e. without a prefix like www, don't work, however, so you have to rely on DNS-level redirecting.)
• For added utility, SSH in with dotcloud ssh and install Drush. (Easiest way I found was to put a symlink to the executable in ~/bin.)

The main outstanding issue is that friendly URL's don't work, because of the nginx configuration. I hope to figure this out soon.

Some other issues and considerations:

• The platform is still in beta, so I experienced a number of API timeouts yesterday. I mentioned this on Twitter and they said they're working on it; today I had fewer timeouts.
• The server instances don't give you root access. They come fully configured but you're locked into your home directory, like shared hosting. I understand the point here - if you changed the server stack, their API and scaling methodologies wouldn't work - but it means if something in the core server config is wrong, tough luck.
• The shell (bash in Ubuntu 10.04 in my examples) is missing a Git client, vim, and nano, and some of its configuration (for vi for instance) is wonky out of the box.
• The intended deployment direction is one-way, from a local dev environment to the servers, so if you change files on the server, you need to rsync them down. (You can SSH in with the dotcloud app and put on a normal SSH key for rsync.)
• Because the webroot is a symlink, any uploaded files have to be outside the webroot (as a symlink as well). This is normal on Capistrano setups, but unusual for most Drupal sites (running on shared or VPS hosting).
• It's free now, but only because it's in beta and they haven't announced pricing. It is yet to be seen if this will be cost-effective when it goes out of beta.
• They promise automated scaling, but it's not clear how that actually works. (Nowhere in the process is there a choice of RAM capacity, for instance.) Does scaling always involve horizontally adding small instances, and if so, does that make sense for high-performance applications?

Conclusion so far

The promise of automated server creation and code deployment is very powerful, and this kind of platform could be perfect for static sites, daemons, or some custom apps. If/when I get it working in Drupal, it could be as simple as a shell script to create a whole live Drupal site from nothing in a few seconds.

Try it out! I'm very curious what others' experience or thoughts are on this approach. I'd especially love for someone to post a solution to the nginx-rewrite issue in the comments.

Update 4/22:

Their support staff recommended reducing the nginx.conf file to one line:

try_files $uri $uri/ /index.php?q=$uri;

And that worked. However, it leaves out all the other recommended rules for caching time, excluding private directories, etc. I asked about these and am still waiting for a reply.

Also, to get file uploads to work properly, you'll need to put your files directory outside of the webroot, and symlink sites/default/files (or equivalent) to that directory, using a postinstall script. Mine looks like this (after creating a ~/drupal-files directory):

chmod g+w /home/dotcloud/current/sites/default && \
ln -s /home/dotcloud/drupal-files /home/dotcloud/current/sites/default/files && \
echo "Symlink for files created."

That runs whenever you run dotcloud push, and is similar to sites deployed with Capistrano, where the same kind of symlink approach is generally used.