Google Distro Trends, 2009/04/23

Just mentioned it in the comments of my previous post, but it’s worth a more public reminder…

Google Distro Trends, 2009/04/23

That’s what I would describe as “the hopes and dreams of a generation” (of Software Freedom lovers), and is the result of fantastic product definition, branding, genuine user excitement, years of incredible — and largely unsung — work of thousands of Debian developers (not to mention all the upstreams)… and a pretty substantial X factor. ;-)

Update: Whenever I talk about this chart in presentations, I always follow up with another chart which puts that incredible rising line in context: Ubuntu vs. Firefox. Interestingly, that chart is looking far less impressive today than it was last year. Which is great news… The rise and rise of Ubuntu is catching up to the star power of Firefox! Wow!

Google Firefox Trends, 2009/04/23

Friends, partners, lovers, warriors…

Goodness me, Pipka and I were married four years ago today. What an adventure! It’s hard to believe how much we’ve achieved together in the last four years. Some musings…

It feels like ages ago that Pia started a company, named in such a way that I would eventually have to join… and join I did. It has been a wonderful experience, not just because it has allowed us to do great things for good money — our preferred arrangement of adjectives! — but because working together has taught us so much about business, the Open Source and ICT industries, and ourselves.

Waugh Partners

One of the nice things about the way we work is that we’ve had plenty of room for our other loves, in both our commercial and community realms. That flexibility, in turn, has changed our business. So, I’ve sunk my teeth into more technical stuff of late, while Pia has been doing lots of education related work. The up-and-down nature of consulting can sometimes be a bit scary, but the ability to evolve our business with our interests — and the changing needs of the market! — is awesome. We’re a rockin’ team.

Of course, none of what I’ve written above really helps to explain why, in the last couple of weeks, we decided to completely change everything. I suppose the flexibility bit might. Anyway, when Pia was invited to join Senator Kate Lundy’s team as a policy advisor, focusing on ICT (as an engine, not a sector), government openness and transparency, and online engagement, there was barely a heartbeat between the two of us before “yes” was the obvious answer. Pia started a week after the offer.

See, there’s a side of me that would totally take a job removing staples from parliamentary dishwashing memos in order to work in Parliament House, but Pia is not the politics tragic that I am… She’s leaping in because the recipe is right: not only are these fantastic and important things to work on, but by working with (seriously clueful) Senator Lundy, there’s a huge opportunity to actually Get Shit Done. A mission worthy of the kung fu steamroller, don’t you think? :-)

Pia’s commute to her new office might have been more of a concern had we not bought a house and moved to Yass just before Christmas. The whole thing is surreal to me on a number of fronts: (a) we bought a house, which seems somewhat shocking among our generation, (b) we decided to buy a house in a matter of days and moved in a matter of weeks, (c) we live in Yass, and (d) we’re really, really enjoying it.

Yeo Cres, Yass backyard

It’s so… serene. We get to spend more time with Pia’s family. There are cows on the hill behind our house. We have a back yard and a shed. The abject lack of urban convenience is improving our abject lack of routine. Win!

So, as of this anniversary we can announce that we’re taking the next logical step for the Waugh family…

We’re getting a puppy!

Happy fourth, poo. :-)

Pia Waugh, post-wedding at the Liberty Theatre

approx: Package caching for Ubuntu (and Debian) lovers

If you have quite a few Debian or Ubuntu machines and you want an easy way to share and speed up package downloads — or reduce your bandwidth consumption — then approx is for you! It’s incredibly easy to set up, but there are a few tricks I’ve learned which can make it especially good for Ubuntu lovers.

I suppose the obvious starting point would be…

apt-get install approx

… and that’s almost all there is to it, but let’s fancy it up a bit.

In /etc/approx/approx.conf:

# use my ISP's ubuntu mirror for most packages
# use central mirrors for security, partner and ports
ubuntu     http://mirror.internode.on.net/pub/ubuntu/ubuntu
secure     http://security.ubuntu.com/ubuntu
partner    http://archive.canonical.com/ubuntu
ports      http://ports.ubuntu.com/ubuntu-ports
medibuntu  http://packages.medibuntu.org

# you can cache PPA packages too, see the sources.list example below
ppa        http://ppa.launchpad.net

# ISP's debian mirror, particularly handy for getting sources
debian     http://mirror.internode.on.net/pub/debian

Demonstrative examples from /etc/apt/sources.list:

# get most packages from my ISP
deb http://approx:9999/ubuntu jaunty main restricted universe

# get security packages from my ISP *or* central
deb http://approx:9999/ubuntu jaunty-security main restricted universe
deb http://approx:9999/secure jaunty-security main restricted universe

# partner and medibuntu packages for naughty stuff we need
deb http://approx:9999/partner jaunty partner
deb http://approx:9999/medibuntu jaunty free non-free

# use generic PPA repository to support arbitrary PPAs!
deb http://approx:9999/ppa/jdub/ppa/ubuntu jaunty main
deb http://approx:9999/ppa/gwibber-team/ubuntu intrepid main

Now you can configure all of your Ubuntu (or Debian) machines to grab their package lists and packages from your approx cache. No more wasted bandwidth when you’re updating, and no more wasted time when you’re stuck watching an update — this is particularly awesome for autmated installs, of course! :-)

Other handy tidbits:

  • You can use the cache to share source packages too — just change ‘deb’ to ‘deb-src’ at the beginning of the line, as usual.
  • approx comes with daily and weekly cronjobs to keep your cache tidy.
  • If you have a bunch of packages lying around in /var/cache/apt/archives which could be put to good use in your approx cache, just use approx-import. Very sweet.

Gosh I’m happy that someone has finally nailed this… I’m pretty sure I’ve tried every attempt to implement a dynamic apt proxy/cache that ever was. Interestingly, approx is yet another OCaml success story. Cool. :-)

Enjoy!

I can finally call my website a blog…

… because it includes pictures of dreadfully cute kittens… yay!

This as-yet-unnamed specimen is destined to be a farm cat — yes, shock, a working cat… catching mice, vermin, etc. — but he had some time in between drinking milk and discovering the world to visit us today. There are a few more shots in my flickr feed, but no LOLcats photos sadly, as we can’t locate our invisible bicycle.


Tabby Kitten by jdubflickr

Slow motion sneeze as scare campaign art

Awesome new ad from the South Australian Government that combines artfully executed slow motion video with some of the tell-tale signs of scare campaign advertising tactics…

“There’s a lot of competition to get attention so you need to have a very stark message, a sharp message, and I think this ad, in a very short amount of time, gets the message across in a very graphic way.”

— SA Health Minister, John Hill

Replacing Apache with nginx for static file serving

This chart represents the regular activity of an Apache server from Monday to Thursday — a massive spike of non-idle processes starting around lunch time — and then an eerily quiet Friday. No, it wasn’t a public holiday, it’s just evidence of win. Notice the gaps late on the 2nd and early on the 3rd? Happy infrastructure hacking in the witching hours!

nginx vs. apache: open/sending

The problem with Apache processes is that they’re big: they chug a lot of memory, and take a long time to get started. Sure, Apache includes a number of slightly more modern process models than the traditional prefork MPM, unfortunately (and predictably enough), PHP keeps us mired in the land of suckage. So when it comes to memory abuse, I was delighted with the results of putting nginx in front of Apache, and giving it complete responsibility for static files:

nginx vs. apache: memory

Reduced memory usage, increased capacity for OS level disk caching, and of course, consistency. Consistency wins every time. Any sysadmin who has had to react to a sudden change in usage patterns will grok the importance of consistency — after all, it’s not being slashdotted or dugg that will kill you, it’s the hardware and software configuration designed for your everyday load.

That’s all very good for Mr. Server Administrator, but nothing is truly measured until we understand the impact on users. Friday lunch time? Business as usual…

nginx vs. apache: traffic

… maaaaybe just a little quicker if they were paying attention. Not that you can tell from this chart. ;-)

So, I’m pretty happy with nginx sitting in front of Apache. It totally changes the performance characteristics of the server (and website!) for the better, while allowing me to fall back on somewhat more mature and extensive Apache features when I need to. For instance, nginx’s rewrite module is pretty good, but Apache’s mod_rewrite is truly the original and the best (read: most insane). Surprisingly, nginx doesn’t support old school CGI, but I suppose most of the cool kids using it are opting for better process models such as FastCGI and proxied application servers anyway.

The biggest win in this instance was the conversion of a caching mod_proxy frontend — lot of static files, inefficiently served by Apache — to the new, inbuilt caching module in nginx 0.7. Initially I had tried the apparently popular ncache — must be a lot of people talking about it hypothetically, methinks — but it is probably best described as an “out-of-tree design, configuration and implementation fail”. Once I sucked up the fear-of-trunk and tried the caching support in nginx 0.7, there was no looking back.

(Finally, a quick shout-out for collectd 4.6.2 and the relatively new collection3 web frontend for it… sadly, I switched to it after these charts were saved, so you don’t get to see the new hotness. Maybe you should try it on your server?)