Be the signal

Friends, partners, lovers, warriors…

April 17, 2009 – 17:00

Goodness me, Pipka and I were married four years ago today. What an adventure! It’s hard to believe how much we’ve achieved together in the last four years. Some musings…

It feels like ages ago that Pia started a company, named in such a way that I would eventually have to join… and join I did. It has been a wonderful experience, not just because it has allowed us to do great things for good money — our preferred arrangement of adjectives! — but because working together has taught us so much about business, the Open Source and ICT industries, and ourselves.

One of the nice things about the way we work is that we’ve had plenty of room for our other loves, in both our commercial and community realms. That flexibility, in turn, has changed our business. So, I’ve sunk my teeth into more technical stuff of late, while Pia has been doing lots of education related work. The up-and-down nature of consulting can sometimes be a bit scary, but the ability to evolve our business with our interests — and the changing needs of the market! — is awesome. We’re a rockin’ team.

Of course, none of what I’ve written above really helps to explain why, in the last couple of weeks, we decided to completely change everything. I suppose the flexibility bit might. Anyway, when Pia was invited to join Senator Kate Lundy’s team as a policy advisor, focusing on ICT (as an engine, not a sector), government openness and transparency, and online engagement, there was barely a heartbeat between the two of us before “yes” was the obvious answer. Pia started a week after the offer.

See, there’s a side of me that would totally take a job removing staples from parliamentary dishwashing memos in order to work in Parliament House, but Pia is not the politics tragic that I am… She’s leaping in because the recipe is right: not only are these fantastic and important things to work on, but by working with (seriously clueful) Senator Lundy, there’s a huge opportunity to actually Get Shit Done. A mission worthy of the kung fu steamroller, don’t you think?

Pia’s commute to her new office might have been more of a concern had we not bought a house and moved to Yass just before Christmas. The whole thing is surreal to me on a number of fronts: (a) we bought a house, which seems somewhat shocking among our generation, (b) we decided to buy a house in a matter of days and moved in a matter of weeks, (c) we live in Yass, and (d) we’re really, really enjoying it.

It’s so… serene. We get to spend more time with Pia’s family. There are cows on the hill behind our house. We have a back yard and a shed. The abject lack of urban convenience is improving our abject lack of routine. Win!

So, as of this anniversary we can announce that we’re taking the next logical step for the Waugh family…

We’re getting a puppy!

Happy fourth, poo.

By Jeff Waugh | Posted in General | Tagged anniversary, friends, kate lundy, lovers, partners, warriors, Waugh Family, Waugh Partners, yass | Comments (15)

approx: Package caching for Ubuntu (and Debian) lovers

April 13, 2009 – 04:51

If you have quite a few Debian or Ubuntu machines and you want an easy way to share and speed up package downloads — or reduce your bandwidth consumption — then approx is for you! It’s incredibly easy to set up, but there are a few tricks I’ve learned which can make it especially good for Ubuntu lovers.

I suppose the obvious starting point would be…

apt-get install approx

… and that’s almost all there is to it, but let’s fancy it up a bit.

In /etc/approx/approx.conf:

# use my ISP's ubuntu mirror for most packages
# use central mirrors for security, partner and ports
ubuntu     http://mirror.internode.on.net/pub/ubuntu/ubuntu
secure     http://security.ubuntu.com/ubuntu
partner    http://archive.canonical.com/ubuntu
ports      http://ports.ubuntu.com/ubuntu-ports
medibuntu  http://packages.medibuntu.org

# you can cache PPA packages too, see the sources.list example below
ppa        http://ppa.launchpad.net

# ISP's debian mirror, particularly handy for getting sources
debian     http://mirror.internode.on.net/pub/debian

Demonstrative examples from /etc/apt/sources.list:

# get most packages from my ISP
deb http://approx:9999/ubuntu jaunty main restricted universe

# get security packages from my ISP *or* central
deb http://approx:9999/ubuntu jaunty-security main restricted universe
deb http://approx:9999/secure jaunty-security main restricted universe

# partner and medibuntu packages for naughty stuff we need
deb http://approx:9999/partner jaunty partner
deb http://approx:9999/medibuntu jaunty free non-free

# use generic PPA repository to support arbitrary PPAs!
deb http://approx:9999/ppa/jdub/ppa/ubuntu jaunty main
deb http://approx:9999/ppa/gwibber-team/ubuntu intrepid main

Now you can configure all of your Ubuntu (or Debian) machines to grab their package lists and packages from your approx cache. No more wasted bandwidth when you’re updating, and no more wasted time when you’re stuck watching an update — this is particularly awesome for autmated installs, of course!

Other handy tidbits:

You can use the cache to share source packages too — just change ‘deb’ to ‘deb-src’ at the beginning of the line, as usual.
approx comes with daily and weekly cronjobs to keep your cache tidy.
If you have a bunch of packages lying around in /var/cache/apt/archives which could be put to good use in your approx cache, just use approx-import. Very sweet.

Gosh I’m happy that someone has finally nailed this… I’m pretty sure I’ve tried every attempt to implement a dynamic apt proxy/cache that ever was. Interestingly, approx is yet another OCaml success story. Cool.

Enjoy!

By Jeff Waugh | Posted in General | Tagged approx, apt, apt-get, cache, Debian, launchpad, packages, ppa, proxy, Ubuntu | Comments (10)

I can finally call my website a blog…

April 11, 2009 – 17:36

… because it includes pictures of dreadfully cute kittens… yay!

This as-yet-unnamed specimen is destined to be a farm cat — yes, shock, a working cat… catching mice, vermin, etc. — but he had some time in between drinking milk and discovering the world to visit us today. There are a few more shots in my flickr feed, but no LOLcats photos sadly, as we can’t locate our invisible bicycle.

Tabby Kitten by jdubflickr

By Jeff Waugh | Posted in General | Tagged blog, cat, kitten, tabby, website | Comments (6)

Slow motion sneeze as scare campaign art

April 6, 2009 – 17:17

Awesome new ad from the South Australian Government that combines artfully executed slow motion video with some of the tell-tale signs of scare campaign advertising tactics…

“There’s a lot of competition to get attention so you need to have a very stark message, a sharp message, and I think this ad, in a very short amount of time, gets the message across in a very graphic way.”

— SA Health Minister, John Hill

http://www.abc.net.au/reslib/200904/r357181_1644198.flv

By Jeff Waugh | Posted in General | Tagged advertising, bullet time, flu, influenza, john hill, scare campaign, slow motion, sneeze, south australia, video | Comments (6)

Replacing Apache with nginx for static file serving

April 6, 2009 – 00:31

This chart represents the regular activity of an Apache server from Monday to Thursday — a massive spike of non-idle processes starting around lunch time — and then an eerily quiet Friday. No, it wasn’t a public holiday, it’s just evidence of win. Notice the gaps late on the 2nd and early on the 3rd? Happy infrastructure hacking in the witching hours!

nginx vs. apache: open/sending

The problem with Apache processes is that they’re big: they chug a lot of memory, and take a long time to get started. Sure, Apache includes a number of slightly more modern process models than the traditional prefork MPM, unfortunately (and predictably enough), PHP keeps us mired in the land of suckage. So when it comes to memory abuse, I was delighted with the results of putting nginx in front of Apache, and giving it complete responsibility for static files:

nginx vs. apache: memory

Reduced memory usage, increased capacity for OS level disk caching, and of course, consistency. Consistency wins every time. Any sysadmin who has had to react to a sudden change in usage patterns will grok the importance of consistency — after all, it’s not being slashdotted or dugg that will kill you, it’s the hardware and software configuration designed for your everyday load.

That’s all very good for Mr. Server Administrator, but nothing is truly measured until we understand the impact on users. Friday lunch time? Business as usual…

nginx vs. apache: traffic

… maaaaybe just a little quicker if they were paying attention. Not that you can tell from this chart.

So, I’m pretty happy with nginx sitting in front of Apache. It totally changes the performance characteristics of the server (and website!) for the better, while allowing me to fall back on somewhat more mature and extensive Apache features when I need to. For instance, nginx’s rewrite module is pretty good, but Apache’s mod_rewrite is truly the original and the best (read: most insane). Surprisingly, nginx doesn’t support old school CGI, but I suppose most of the cool kids using it are opting for better process models such as FastCGI and proxied application servers anyway.

The biggest win in this instance was the conversion of a caching mod_proxy frontend — lot of static files, inefficiently served by Apache — to the new, inbuilt caching module in nginx 0.7. Initially I had tried the apparently popular ncache — must be a lot of people talking about it hypothetically, methinks — but it is probably best described as an “out-of-tree design, configuration and implementation fail”. Once I sucked up the fear-of-trunk and tried the caching support in nginx 0.7, there was no looking back.

(Finally, a quick shout-out for collectd 4.6.2 and the relatively new collection3 web frontend for it… sadly, I switched to it after these charts were saved, so you don’t get to see the new hotness. Maybe you should try it on your server?)

By Jeff Waugh | Posted in General | Tagged Apache, ncache, nginx, performance, web development, web servers | Comments (22)

Case-insensitive mapping with mod_rewrite’s RewriteMap

March 29, 2009 – 03:40

Sometimes when you need to manage a massive pile of URL redirections — for instance, when you’re playing snatch-the-tablecloth with your web platform — it’s handy to mash them all together with mod_rewrite’s RewriteMap feature.

I hit a frustrating stumbling block with a recent project, however: What happens if you need your RewriteMap key to be case-insensitive? This is incredibly likely if you’re migrating away from an undisciplined Windows-based web platform.

It was immediately obvious that I’d have to match against consistent casing, but not at all obvious how this would work in the context of mod_rewrite. After some trial and error, here’s what I came up with:

RewriteMap lowercase int:tolower
RewriteMap urls dbm:/srv/www.example.com/urls.db
RewriteCond %{REQUEST_URI} ^(/.*\.html)$
RewriteCond ${lowercase:%1} ^(/.*\.html)$
RewriteCond ${urls:%1|NOT_FOUND} !NOT_FOUND
RewriteRule ^(/.*\.html)$ http://%{HTTP_HOST}${urls:%1}? [R=permanent,L]

Here’s how it works:

First we set up the built in lowercase translation rewrite map… handy!
Then we define our great big URL map, which is just a list of lowercase URLs (yes, they’re lowercased in the file) and their destinations.
Make sure we’re only mucking around with html URLs, but also grab the REQUEST_URI for use further along (see the parentheses in the second parameter).
We use Apache’s handy internal lowercase map to convert the previous line’s match to lowercase, and we grab the whole thing again… but this time, what we’re grabbing is the output of the lowercase map.
Tricky little hack to make sure our URL is represented in the redirection map. If there’s no match, the default value returned is NOT_FOUND… but we only continue to the next line if the map does not return NOT_FOUND.
Finally, permanently redirect to the destination URL, as matched against the lowercase original.

Rock and roll.

By Jeff Waugh | Posted in General | Tagged Apache, case-insensitive, mod_rewrite, rewritecond, rewritemap, rewriterule | Comments (3)

Devils and Penguins

March 18, 2009 – 15:35

Tux passes the baton to Tuz, the hastily-disguised Tasmanian Devil… at least for this release. Great to see one of the more high profile pledges of the madcap, ridiculous, wonderful and incredibly generous linux.conf.au 2009 auction being paid off.

Tuz

linux.conf.au 2009: Tuz

Impact: help prevent extinction of species

The Tasmanian Devil is a shy iconic Australian creature named for its
spine-chilling screech.  It is threatened with extinction due to a
scientifically interesting but horrific transmissible facial cancer.

This one is standing in for Tux for one release using the far less-known
Devil Facial Tux Disguise.

Save The Tasmanian Devil http://tassiedevil.com.au

Signed-off-by: Linux.conf.au Hobart Team <contact@marchsouth.org>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

By Jeff Waugh | Posted in General | Tagged Australia, cancer, kernel, lca2009, linus torvalds, Linux, linux.conf.au, rusty russell, tasmania, tasmanian devil, tux, tuz | Comments (4)

QotD: Guy Rundle

March 11, 2009 – 13:59

When your movement is being led by a 13-year-old child, the morbidly obese recovering benzodiazepine addict radio star Rush Limbaugh, and a mystical simpleton like Joe the Plumber, then you’re at the point medieval Europe reached around the time of the children’s crusade.

— Guy Rundle, US right on a long march to medievalism

By Jeff Waugh | Posted in Quote of the Day | Tagged conservative, cpac, gop, guy rundle, joe the plumber, jonathan krohn, republican, rush limbaugh | Comments (7)

Joining the Blackout

February 23, 2009 – 11:42

I’m joining the Internet blackout today, and not just for my New Zealand friends who are combating this draconian copyright fundamentalism… I’m joining it because if this kind of legislation succeeds in New Zealand — who have a pretty good history of sticking it to The Man — it could succeed anywhere.

Here’s a great cartoon about the problematic legislation from Scoop which very neatly explains the issue:

Update: New Zealand Prime Minister John Key has announced that S92A will go on hold until March 27, apparently hoping that an agreement will be reached on a voluntary code of practice. The legislation isn’t dead, but this is more than “just a flesh wound”.

Congratulations to the Creative Freedom Foundation for a well-organised, multi-facted and effective campaign against S92A… thanks also to Brenda and Nat, who worked tirelessly to hyper-connect Kiwis and their friends around the world.

By Jeff Waugh | Posted in General | Tagged blackout, brenda wallace, copyright, creative freedom, internet blackout, john key, nat torkington, new zealand, s92a | Comments (7)

QotD: John McCain’s favourite joke

February 10, 2009 – 01:48

I feel terrible for all the mothers in the state of Arizona because, as you know: Barry Goldwater from Arizona ran for President of the United States, Morris Udall from Arizona ran for President of the United States, Bruce Babbitt from Arizona ran for President of the United States, and I, John McCain from Arizona ran for President of the United States… Arizona may be the only state in the nation where mothers can no longer tell their children that some day they can grow up and be President of the United States.

— John McCain in May 2003, via ZUG’s 100 Senator Prank