Watching nginx upstreams with collectd

Already happy with nginx in front of Apache for a number of sites, I decided it was time to start testing nginx/fastcgi on my personal server (the serial crash test dummy of my web operations). The only problem: I have yet to find a sensible method of grabbing useful runtime information from the PHP fastcgi process itself, and if you can’t sensibly watch it, you can’t sensibly deploy it.

So for now, instead of watching the PHP fastcgi process directly, I’m tracking its performance and usage from nginx’s perspective. You can log all kinds of data about upstream performance with nginx:

log_format upstream '$remote_addr - - [$time_local] "$request" $status '
    'upstream $upstream_response_time request $request_time '
    '[for $host via $upstream_addr]';

Then we log to a central upstream.log file from every location block which includes a fastcgi_pass parameter. For example:

location ~ \.php$ {
    include  fastcgi_params;
    access_log  /var/log/nginx/upstream.log  upstream;
    fastcgi_pass  fcgi_php;
    fastcgi_param  SCRIPT_FILENAME  $wordpress_root$fastcgi_script_name;

Now we know how many requests the PHP fastcgi process is handling, and how quickly it’s doing so. collectd‘s tail plugin can watch this log file…

<Plugin tail>
  <File "/var/log/nginx/upstream.log">
  Instance "nginx"
      Regex ".*"
      DSType "CounterInc"
      Type counter
      Instance "requests"
      Regex " upstream ([0-9.]*) "
      DSType GaugeAverage
      Type delay
      Instance "upstream"

… and turn it into something readable. First, the number of requests per second (which I only started watching at 14:30 this afternoon), then the delay for each request:

nginx Upstream Requests

nginx Upstream Response

(Relatively boring statistics here, as it’s only monitoring the dynamic processing of my personal sites.)

Combining nginx’s flexible logging and collectd’s tail plugin makes it pretty easy to watch the usage and performance of whatever you’re running behind nginx, even if you can’t instrument the application itself.

… and thus far, I’m pretty happy with the performance, reliability and resource usage of nginx in front of PHP in fastcgi mode. :-)

WordPress and Drizzle

So, for amusement, education and a desire to put Drizzle through its paces with a real-world application, I built a Drizzle database adapter for WordPress.

Rather than completely dumping the native wpdb class (as most WordPress database adapters appear to do), I have subclassed it, replacing only the methods which use mysql_ functions.

This way, it’s easier to maintain, and more likely to work with plugins and future versions of WordPress. I have also written a cheesy, retroactive test suite for wpdb… I haven’t done a lot of TDD, so I have no idea if it’s any good, but it was helpful during development. ;-)

Because Drizzle has removed column types and various bits of syntax that WordPress (and heaps of  plugins) rely on, the adapter does a tiny bit of query munging along the way. For now, the only filters required are for CREATE, ALTER and friends.

Here’s a screenshot of WordPress running on Drizzle. Note that in the sidebar, I’m showing off two plugins which maintain their own tables — WP-PostRatings and Twitter Tools.

WordPress on Drizzle

To test this bad boy out — because it’s absolutely not for production blogs! — you’ll need the Drizzle server (and its dependencies, naturally), the Drizzle PHP extension, WordPress 2.8 and my adapter. Note that I have yet to test it with WordPress MU, but it shouldn’t require many changes if it doesn’t work already.

If you use Ubuntu and want an easier time of it, you can get almost everything from the following PPAs. The Drizzle server isn’t packaged, but it’s incredibly easy to build and changes so quickly that you’ll probably find you want to keep it up to date from the source anyway.

deb hardy main
deb hardy main

(I have only built my packages for Ubuntu 8.04, hardy, but you can always just grab the source packages and build them for whatever you’re using.)

Make sure you install the database adapter before you go through the WordPress install process. In every other respect, your test blog should operate in a completely unsurprising manner. Except when Drizzle crashes… but that’s precisely why you’re testing it, right? ;-)

On Drizzle

I am incredibly impressed with the Drizzle project. It’s a living case study of Open Source innovation and project renewal.

In response to what could be regarded as the unadventurous maturity of the MySQL project, the Drizzle developers have not simply chosen a new goal and forked the code — they have crafted a mission based on an insightful reading of current and future needs, questioned everything about MySQL without throwing away what they had learned, and thoroughly redefined their expectations and model for community collaboration.

It’s the perfect application of Software Freedom’s most functional of permissions: the freedom to fork. Like other forks built on both technical and social foundations — such as Firefox and, coincidentally, WordPress — I think it will eclipse its predecessor. Yes, even the mighty MySQL. :-)

I hope that what the Drizzle developers have done will teach and inspire other projects to look beyond their own horizon.

Here’s a rocktastic presentation by Brian Aker, “Drizzle: Rethinking MySQL for the Web”, from the recent Open Source Bridge conference:

[ ?posts_id=2312282&dest=-1]