Comments on: Watching nginx upstreams with collectd http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/ where we're going, we don't need roads... Sat, 28 Jan 2012 02:14:19 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Michael Shigorin http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4769 Michael Shigorin Mon, 29 Mar 2010 20:57:58 +0000 http://bethesignal.org/?p=1577#comment-4769 Then you probably don't need 150 rrds per server anyways. OTOH one also might be interested in Clustrx which is an emerging aggregator capable of handling thousands of nodes with dozens of parameters and resolution down to a second. It is being developed for HPC environments and should be available this year. Then you probably don’t need 150 rrds per server anyways.

OTOH one also might be interested in Clustrx which is an emerging aggregator capable of handling thousands of nodes with dozens of parameters and resolution down to a second. It is being developed for HPC environments and should be available this year.

]]> By: Michael Shigorin http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4768 Michael Shigorin Mon, 29 Mar 2010 20:53:47 +0000 http://bethesignal.org/?p=1577#comment-4768 How, is it *that* Jeff Waugh complaining about IO of something capable of networked aggregation? :) We run a few servers for non-commercial FLOSS related projects, and having the less powerful system handle collectd streams (and hopefully logs in the future when we do VPN as well) helps immensely. The other way around might be dedicated spindle for logs/rrds and backups. How, is it *that* Jeff Waugh complaining about IO of something capable of networked aggregation? :)

We run a few servers for non-commercial FLOSS related projects, and having the less powerful system handle collectd streams (and hopefully logs in the future when we do VPN as well) helps immensely. The other way around might be dedicated spindle for logs/rrds and backups.

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4483 Jeff Waugh Tue, 11 Aug 2009 01:29:03 +0000 http://bethesignal.org/?p=1577#comment-4483 @Ngo Ky Lam: <a href="http://collectd.org/" rel="nofollow">collectd</a> itself includes a couple of web frontends which render the charts for you on demand. @Ngo Ky Lam: collectd itself includes a couple of web frontends which render the charts for you on demand.

]]> By: Ngo Ky Lam http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4482 Ngo Ky Lam Mon, 10 Aug 2009 21:31:45 +0000 http://bethesignal.org/?p=1577#comment-4482 Hi, may i ask what kind of script do you use to generate collectd graphs ? Hi, may i ask what kind of script do you use to generate collectd graphs ?

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4450 Jeff Waugh Wed, 05 Aug 2009 00:16:11 +0000 http://bethesignal.org/?p=1577#comment-4450 @Astro: Say you've got 150 rrd files per server. Say you've got 100 servers. Say you want them all reporting collectd data to a single server. Now you've got one machine essentially doing random writes over 15000 disk locations every 10 seconds. (Sure, there are controls for this if you want to risk your stats to memory, which is not always a bad idea, but still... it's something the server admin needs to consider.) @Astro: Say you’ve got 150 rrd files per server. Say you’ve got 100 servers. Say you want them all reporting collectd data to a single server. Now you’ve got one machine essentially doing random writes over 15000 disk locations every 10 seconds.

(Sure, there are controls for this if you want to risk your stats to memory, which is not always a bad idea, but still… it’s something the server admin needs to consider.)

]]> By: Astro http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4449 Astro Wed, 05 Aug 2009 00:08:59 +0000 http://bethesignal.org/?p=1577#comment-4449 I've got some good words to lose on collectd. I peeked into its source and found very clean C code indeed. The architecture is quite smart, too: agents only collect data from the local system and send UDP packets to the servers with the rrdtool plugin enabled. That way I assume there is very little chance for a remote vulnerability in the agents. collectd gathers all data by its native C plugins. Contrary to munin, they don't fork other processes and eat therefore only very few system resources. However, I don't like the web front-ends of collectd (collection3). It's very basic, difficult to configure and I really hope something better will come up in the future. Jeff Waugh: what's wrong with I/O in collectd? Maybe a cache misconfiguration in the rrdtool plugin? I’ve got some good words to lose on collectd. I peeked into its source and found very clean C code indeed. The architecture is quite smart, too: agents only collect data from the local system and send UDP packets to the servers with the rrdtool plugin enabled. That way I assume there is very little chance for a remote vulnerability in the agents.

collectd gathers all data by its native C plugins. Contrary to munin, they don’t fork other processes and eat therefore only very few system resources.

However, I don’t like the web front-ends of collectd (collection3). It’s very basic, difficult to configure and I really hope something better will come up in the future.

Jeff Waugh: what’s wrong with I/O in collectd? Maybe a cache misconfiguration in the rrdtool plugin?

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4397 Jeff Waugh Thu, 23 Jul 2009 15:12:37 +0000 http://bethesignal.org/?p=1577#comment-4397 The biggest PITA with collectd is I/O. The biggest PITA with collectd is I/O.

]]> By: Paul Shirren http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4396 Paul Shirren Thu, 23 Jul 2009 11:55:02 +0000 http://bethesignal.org/?p=1577#comment-4396 Cherokee would keep up ok, but then the next version would come out and break you config file. Nobody using php-fpm? It takes a bit more work patching php and applying suhosin etc instead of letting a distro do it. Hopefully it will be packaged or part of standard php one day. Curious about collectd memory & cpu being c based vs one of the usual suspects (cacti, munin etc) - every byte counts on a vps. Do you notice it? Cherokee would keep up ok, but then the next version would come out and break you config file. Nobody using php-fpm? It takes a bit more work patching php and applying suhosin etc instead of letting a distro do it. Hopefully it will be packaged or part of standard php one day. Curious about collectd memory & cpu being c based vs one of the usual suspects (cacti, munin etc) – every byte counts on a vps. Do you notice it?

]]> By: Jones Lee http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4394 Jones Lee Thu, 23 Jul 2009 04:20:40 +0000 http://bethesignal.org/?p=1577#comment-4394 I'd love to see Cherokee + fcgi vs nginx + fcgi result. I’d love to see Cherokee + fcgi vs nginx + fcgi result.

]]> By: Jeff Waugh http://bethesignal.org/blog/2009/07/22/watching-nginx-upstreams-with-collectd/#comment-4390 Jeff Waugh Wed, 22 Jul 2009 23:44:20 +0000 http://bethesignal.org/?p=1577#comment-4390 Either way, you should always use an opcode cache (APC appears to be the most stable and attentively maintained). :-) And yes, I'm using spawn-fcgi to manage the PHP fastcgi processes. If you don't depend on Apache features, I would now recommend going down the fastcgi route. I haven't done it for so long because Apache has been something of a familiar security blanket. :-) Either way, you should always use an opcode cache (APC appears to be the most stable and attentively maintained). :-) And yes, I’m using spawn-fcgi to manage the PHP fastcgi processes.

If you don’t depend on Apache features, I would now recommend going down the fastcgi route. I haven’t done it for so long because Apache has been something of a familiar security blanket. :-)

]]>