Comments on: Watching nginx upstreams with collectd http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/ where we're going, we don't need roads... Sat, 20 Feb 2010 06:58:05 +1100 http://wordpress.org/?v=2.9.2 hourly 1 By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4483 Jeff Waugh Tue, 11 Aug 2009 01:29:03 +0000 http://bethesignal.org/?p=1577#comment-4483 @Ngo Ky Lam: <a href="http://collectd.org/" rel="nofollow">collectd</a> itself includes a couple of web frontends which render the charts for you on demand. @Ngo Ky Lam: collectd itself includes a couple of web frontends which render the charts for you on demand.

]]> By: Ngo Ky Lam http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4482 Ngo Ky Lam Mon, 10 Aug 2009 21:31:45 +0000 http://bethesignal.org/?p=1577#comment-4482 Hi, may i ask what kind of script do you use to generate collectd graphs ? Hi, may i ask what kind of script do you use to generate collectd graphs ?

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4450 Jeff Waugh Wed, 05 Aug 2009 00:16:11 +0000 http://bethesignal.org/?p=1577#comment-4450 @Astro: Say you've got 150 rrd files per server. Say you've got 100 servers. Say you want them all reporting collectd data to a single server. Now you've got one machine essentially doing random writes over 15000 disk locations every 10 seconds. (Sure, there are controls for this if you want to risk your stats to memory, which is not always a bad idea, but still... it's something the server admin needs to consider.) @Astro: Say you’ve got 150 rrd files per server. Say you’ve got 100 servers. Say you want them all reporting collectd data to a single server. Now you’ve got one machine essentially doing random writes over 15000 disk locations every 10 seconds.

(Sure, there are controls for this if you want to risk your stats to memory, which is not always a bad idea, but still… it’s something the server admin needs to consider.)

]]> By: Astro http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4449 Astro Wed, 05 Aug 2009 00:08:59 +0000 http://bethesignal.org/?p=1577#comment-4449 I've got some good words to lose on collectd. I peeked into its source and found very clean C code indeed. The architecture is quite smart, too: agents only collect data from the local system and send UDP packets to the servers with the rrdtool plugin enabled. That way I assume there is very little chance for a remote vulnerability in the agents. collectd gathers all data by its native C plugins. Contrary to munin, they don't fork other processes and eat therefore only very few system resources. However, I don't like the web front-ends of collectd (collection3). It's very basic, difficult to configure and I really hope something better will come up in the future. Jeff Waugh: what's wrong with I/O in collectd? Maybe a cache misconfiguration in the rrdtool plugin? I’ve got some good words to lose on collectd. I peeked into its source and found very clean C code indeed. The architecture is quite smart, too: agents only collect data from the local system and send UDP packets to the servers with the rrdtool plugin enabled. That way I assume there is very little chance for a remote vulnerability in the agents.

collectd gathers all data by its native C plugins. Contrary to munin, they don’t fork other processes and eat therefore only very few system resources.

However, I don’t like the web front-ends of collectd (collection3). It’s very basic, difficult to configure and I really hope something better will come up in the future.

Jeff Waugh: what’s wrong with I/O in collectd? Maybe a cache misconfiguration in the rrdtool plugin?

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4397 Jeff Waugh Thu, 23 Jul 2009 15:12:37 +0000 http://bethesignal.org/?p=1577#comment-4397 The biggest PITA with collectd is I/O. The biggest PITA with collectd is I/O.

]]> By: Paul Shirren http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4396 Paul Shirren Thu, 23 Jul 2009 11:55:02 +0000 http://bethesignal.org/?p=1577#comment-4396 Cherokee would keep up ok, but then the next version would come out and break you config file. Nobody using php-fpm? It takes a bit more work patching php and applying suhosin etc instead of letting a distro do it. Hopefully it will be packaged or part of standard php one day. Curious about collectd memory & cpu being c based vs one of the usual suspects (cacti, munin etc) - every byte counts on a vps. Do you notice it? Cherokee would keep up ok, but then the next version would come out and break you config file. Nobody using php-fpm? It takes a bit more work patching php and applying suhosin etc instead of letting a distro do it. Hopefully it will be packaged or part of standard php one day. Curious about collectd memory & cpu being c based vs one of the usual suspects (cacti, munin etc) - every byte counts on a vps. Do you notice it?

]]> By: Jones Lee http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4394 Jones Lee Thu, 23 Jul 2009 04:20:40 +0000 http://bethesignal.org/?p=1577#comment-4394 I'd love to see Cherokee + fcgi vs nginx + fcgi result. I’d love to see Cherokee + fcgi vs nginx + fcgi result.

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4390 Jeff Waugh Wed, 22 Jul 2009 23:44:20 +0000 http://bethesignal.org/?p=1577#comment-4390 Either way, you should always use an opcode cache (APC appears to be the most stable and attentively maintained). :-) And yes, I'm using spawn-fcgi to manage the PHP fastcgi processes. If you don't depend on Apache features, I would now recommend going down the fastcgi route. I haven't done it for so long because Apache has been something of a familiar security blanket. :-) Either way, you should always use an opcode cache (APC appears to be the most stable and attentively maintained). :-) And yes, I’m using spawn-fcgi to manage the PHP fastcgi processes.

If you don’t depend on Apache features, I would now recommend going down the fastcgi route. I haven’t done it for so long because Apache has been something of a familiar security blanket. :-)

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4389 Jeff Waugh Wed, 22 Jul 2009 23:41:11 +0000 http://bethesignal.org/?p=1577#comment-4389 collectd includes a set of CGI scripts to browse and render the graphs. collectd includes a set of CGI scripts to browse and render the graphs.

]]> By: Matthias Adler http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4388 Matthias Adler Wed, 22 Jul 2009 21:37:10 +0000 http://bethesignal.org/?p=1577#comment-4388 I've been impressed by nginx so far as well, but it does not have the ability to spawn cgi-processes on its own afaik - so I relied on lighttpd's spawn-fcgi (nowadays external) helper, but I am seriously considering proxying to apache. Some PHP stuff (pecl uploadprogress anyone) more or less only works with apache. So here is my question: Memory consumption issues aside, would you rather recommend apache or running fcgi processes + opcode optimizers (apc, etc) on production sites. Whats your take ? I’ve been impressed by nginx so far as well, but it does not have the ability to spawn cgi-processes on its own afaik - so I relied on lighttpd’s spawn-fcgi (nowadays external) helper, but I am seriously considering proxying to apache. Some PHP stuff (pecl uploadprogress anyone) more or less only works with apache. So here is my question: Memory consumption issues aside, would you rather recommend apache or running fcgi processes + opcode optimizers (apc, etc) on production sites. Whats your take ?

]]>