Watching nginx upstreams with collectd

Already happy with nginx in front of Apache for a number of sites, I decided it was time to start testing nginx/fastcgi on my personal server (the serial crash test dummy of my web operations). The only problem: I have yet to find a sensible method of grabbing useful runtime information from the PHP fastcgi process itself, and if you can’t sensibly watch it, you can’t sensibly deploy it.

So for now, instead of watching the PHP fastcgi process directly, I’m tracking its performance and usage from nginx’s perspective. You can log all kinds of data about upstream performance with nginx:

log_format upstream '$remote_addr - - [$time_local] "$request" $status '
    'upstream $upstream_response_time request $request_time '
    '[for $host via $upstream_addr]';

Then we log to a central upstream.log file from every location block which includes a fastcgi_pass parameter. For example:

location ~ \.php$ {
    include  fastcgi_params;
    access_log  /var/log/nginx/upstream.log  upstream;
    fastcgi_pass  fcgi_php;
    fastcgi_param  SCRIPT_FILENAME  $wordpress_root$fastcgi_script_name;

Now we know how many requests the PHP fastcgi process is handling, and how quickly it’s doing so. collectd‘s tail plugin can watch this log file…

<Plugin tail>
  <File "/var/log/nginx/upstream.log">
  Instance "nginx"
      Regex ".*"
      DSType "CounterInc"
      Type counter
      Instance "requests"
      Regex " upstream ([0-9.]*) "
      DSType GaugeAverage
      Type delay
      Instance "upstream"

… and turn it into something readable. First, the number of requests per second (which I only started watching at 14:30 this afternoon), then the delay for each request:

nginx Upstream Requests

nginx Upstream Response

(Relatively boring statistics here, as it’s only monitoring the dynamic processing of my personal sites.)

Combining nginx’s flexible logging and collectd’s tail plugin makes it pretty easy to watch the usage and performance of whatever you’re running behind nginx, even if you can’t instrument the application itself.

… and thus far, I’m pretty happy with the performance, reliability and resource usage of nginx in front of PHP in fastcgi mode. 🙂

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *