RftE: Python

After finding this gem through Matthias Clasen and Ryan Lortie, I realised that my Ready for the Enterprise series simply had to include negative reviews too: Things that are clearly not Ready for the Enterprise.

Apparently, Arjan van de Ven has declared war on context switches and process wakeups in Red Hat’s bugzilla. A bunch of nice bugs there, and many more to come. Ryan recently wrote about this mess and ways to look for it, too.

The surprise for me was a bug about yum-updatesd — well, okay, that wasn’t much of a surprise — and related atrocities. It turns out that the Python shell wakes up every 100msec. Even when it has nothing to do. Try it at home:

$ python
Python 2.4.3 (#2, Aug 25 2006, 17:37:59)
[GCC 4.1.2 20060817 (prerelease) (Ubuntu 4.1.1-11ubuntu1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

And in another terminal:

$ strace -p $(pidof python)
Process 13787 attached - interrupt to quit
select(1, [0], NULL, NULL, {0, 96000})  = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 100000}) = 0 (Timeout)
select(1, [0], NULL, NULL, {0, 100000}) = 0 (Timeout)
...

Holy crap! With pygobject and threading in the mix, for every long lifecycle Python process left running on your desktop machine — and for Ubuntu users, it really adds up — your CPU is getting a thorough workout while your battery auditions for the next explosive YouTube video meme. Let’s not even start on data centre use cases… Just buy more cooling!

Seriously, can you imagine? How much code do you think they’re running on the Enterprise — even when they’re stuck on emergency power because Wesley got his fly stuck in the reactor core again?

They still had to keep over a thousand people breathing, all their communicators working, critical computing infrastructure functioning, and so on. Reckon they’d sink power and cycles into wasted context switches? Nosiree. Wouldn’t happen. They never had to deal with this shit in Star Trek!

And that is why Python gets our very first Not Ready for the Enterprise review.

I mean, the Enterprise is not like a dump truck. You can’t just put things in it and expect it to work. The Enterprise is more like a series of tubes, and when you get a fat wad of agitated Python stuck in those tubes, you’ve got snakes on a m@#%#rf%!cking spaceship, and that just ain’t right!

Trek to Tubes to SoaP, ftw!

(Completely outrageous examples and fabrication used for effect. No actual Pythons were harmed during the creation of this entry.)

This entry was posted in Ready for the Enterprise and tagged , . Bookmark the permalink.

18 Responses to RftE: Python

  1. Chris says:

    This is misunderstood; the bugzilla link describes a 100ms wakeup in pyGobject, not in Python. The shell waking up is not the same as the interpreter waking up.

  2. jdub says:

    This is misunderstood; the commenter describes fact, not truthiness. The computer science you get from a book is not the same as the knowledge about Python you feel in your gut.

    Like, seriously, c’mon people?

  3. Kevin Wang says:

    Get your facts straight. it’s the readline library that’s doing the select() calls, not python.

    strace python -c ‘while True: pass’

    shows no select() looping every 100ms

  4. jdub says:

    You read that in a book, didn’t you? All fact. No heart.

  5. Mary says:

    I’m back to J2EE on my Death Star software blueprints, you hippie bastards.

  6. Gekitsuu says:

    I’m confused has the m%*&#(%&#@!n bug been assigned to Samuel L Jackson yet or not?

  7. Benno says:

    Ermm, it is python’s readline that is in a select loop, for no real good reason, so despite the humour in which this was posted, I think it is a real problem that shouldn’t be there.

  8. Matt (msw) Wilson says:

    jdub:

    What you’re seeing is a part of the python readline module. readline.c:767 says:

    struct timeval timeout = {0, 100000}; /* 0.1 seconds */

    This .1 second sleep is *only* when you’re sitting at a prompt using the readline module.

  9. Matt (msw) Wilson says:

    Kevin: it’s the python readline module, not the readline library.

  10. bi says:

    tail -f wakes up every one second. That’s why Linux sucks, and why Windows and Mac rock, you tree-hugging Saddam-loving anti-capitalist cows.

  11. Osma says:

    python -c ‘while True: pass’
    File “”, line 1
    ‘while
    ^
    SyntaxError: invalid syntax

    No select(), that’s the truth.

  12. Ian says:

    It is readline, Ruby’s irb produces exactly the same output. I don’t think it’s fair to label Python as not ready for the enterprise, please make a correction.

  13. 100ms says:

    100ms is eternity for a modern PC. In 100ms it can run about 400M cycles (if 4GHz) which is about 340M assembly instructions. Waking up every 340Million instructions to run 200 instructions is zero load. The computer will die of boredom. Not to mention it’s the readline library…

    We are talking about orders of numbers here.

  14. Brantley Harris says:

    I demand Guido apologize to Geraldo.

  15. amk says:

    The link for this comment is to the Python bug for this.

    This is tied to the version of readline being used; the
    select code is only used if HAVE_RL_CALLBACK is defined, and
    a comment in Python’s configure.in claims it’s only
    defined
    with readline 2.1. Current versions of readline are 4.3 and
    5.1; are people still using such an ancient version of
    readline?

  16. Jon Dowland says:

    I’ve observed similar problems in a ruby program. I initially thought it was glade, but it happens in the irb interpreter too. It’s probably readline for the latter, but that doesn’t explain the former. (It wasn’t glade).

  17. Eero Tamminen says:

    On my half-obsolete SUSE 9.1 distro, Python v2.3.3 does not poll select() in shell prompt when straced. Maybe this bug is of more recent origin?

    > 100ms is eternity for a modern PC

    Modern PC is a mobile device with power management in-core and dyntick patched kernel. It can’t sleep at night if reptiles are interrupting it all the time…

Comments are closed.