A couple of weeks ago, while chatting with Rob, I mentioned that John gave me an awesome tip that I use all the time: The pget command in lftp, particularly for sftp downloads. That’s right: Not only does lftp provide a sweet interface for sftp, but it can also make multiple connections and download in parallel, just like it does with ftp.
I’ve used this for ages to take full advantage of my Internet bandwidth when downloading large files from the USA to Australia: 320K/s vs. 1.21M/s is an easy choice to make! So it surprised me when Rob suggested that I only had to do this because “ssh was broken”.
If that were truly the case, I ought to be able to do an http download at maximum speed between the same machines… testing, testing, downloading, testing… Holy crap! ssh is broken!
Rob explained some stuff about windowing, and how ssh had to reimplement things, but was doing so badly… I kinda tuned out at that point, as I was negotiating the stages of grief for my beloved ssh.
So, when a new version of OpenSSH landed in Hardy the other day, citing windowing-related changes, I figured I’d have to check if it made any difference for my use case:
gutsy: 183824384 bytes transferred in 559 seconds (320.9K/s)
hardy: 183824384 bytes transferred in 145 seconds (1.21M/s)
Although I tested on different physical machines on my local network, the performance increase is undeniably due to changes in the new release… I rebuilt it on gutsy and now that machine flies too. 🙂
Ah, the simple things…