A couple of weeks ago, while chatting with Rob, I mentioned that John gave me an awesome tip that I use all the time: The pget command in lftp, particularly for sftp downloads. That’s right: Not only does lftp provide a sweet interface for sftp, but it can also make multiple connections and download in parallel, just like it does with ftp.
I’ve used this for ages to take full advantage of my Internet bandwidth when downloading large files from the USA to Australia: 320K/s vs. 1.21M/s is an easy choice to make! So it surprised me when Rob suggested that I only had to do this because “ssh was broken”.
If that were truly the case, I ought to be able to do an http download at maximum speed between the same machines… testing, testing, downloading, testing… Holy crap! ssh is broken!
Rob explained some stuff about windowing, and how ssh had to reimplement things, but was doing so badly… I kinda tuned out at that point, as I was negotiating the stages of grief for my beloved ssh.
So, when a new version of OpenSSH landed in Hardy the other day, citing windowing-related changes, I figured I’d have to check if it made any difference for my use case:
gutsy: 183824384 bytes transferred in 559 seconds (320.9K/s)
hardy: 183824384 bytes transferred in 145 seconds (1.21M/s)
Although I tested on different physical machines on my local network, the performance increase is undeniably due to changes in the new release… I rebuilt it on gutsy and now that machine flies too. 
Ah, the simple things…
20 Comments
Wow! Does this mean that sshfs mounts will get the same kind of speed improvement?
I’ve always found SSH slow, but I expected this to do due to the encryption and the usage of TCP (instead of UDP)
Still, with my 19-down | 1-up megabit connection, transferring a file over SSH (using nautilus, that is!) to another computer also connected on, is awfully slow. We’re talking hours here for just a movie. 700 mb / 5 hours … 700 * 1000 / 3 * 60 * 60 == 700000 / 18000 == 38, 9 kb/s
So, where the connection is capable of 1000/8 = 125 kb/s upsteam, all I get is 38,9 kb/s on Gutsy. I very much look forward in seeing this speed increase!
Can you post the gutsy packages?
@David: Almost certainly not for interactive speed, but it will very likely have an impact on throughput depending on the link you’re using.
@Duncan Mak: They’re really easy to rebuild:
/etc/apt/sources.listsudo apt-get build-dep opensshwill install the build dependencies for you.apt-get source opensshwill get the source packages for you. You might want to do this in/tmp.cd openssh-4.7p1dpkg-buildpackage -uc -us -rfakerootwill run through the build./tmp, ready to install.Ah! Never thought of putting the next distro’s deb-src line in to do backports! Thanks for the tip.
Or rather than building it yourself, you could use a PPA on Launchpad
Yes, put the deb-src line in your sources.list, but don’t be blind and silly enough to put the binary packages line in as well. Bad Things[tm] might happen. Allegedly.
Oh well, time I moved the lappie to Hardy anyway I guess.
I’ve pretty sure i’ve seen an original print of Faster, Pussycat! Kill! Kill! at the Koru Framing Studio a while ago. Definitely worth checking out if you like vintage movie posters.
So you need the new openssh on both sides of the link to get faster transfers, or is it enough to get a newer ssh client?
@Marius Gedminas: Just the client. My server is running an unmolested gusty… aside from the nickname.
@Meneer: “but I expected this to do due to the encryption and the usage of TCP (instead of UDP)” Why TF would anyone use UDP for SSH (aside form pathological corner cases)?
@Satya
No, it would not be smart to use UDP.
But in general file-transer-protocols that use UDP are a little bit faster than TCP.
Off course SSH is more than just file-transfers. Esspecially the shell stuff sort of requires serialized communications.
All I meant was that I _thought_ the fact that SSH transfer were slow was _because_ of that. Not that using UDP would be a smart thing for SSH. I just assumed it had something to do with that. I was wrong.
FWIW, there has just been an update to openssh, so it’s possible this fix just got bundled out to everyone (possibly with a security update - i don’t have changelogs to hand). Might be worth re-running some tests with the standard packages.
Hopefully this might also fix some of the real problems I’ve had at home across a LAN transferring huge files (eg: 4.7 gig ISO’s of a certain linux distro we’re all familiar with), where it gets between 1-2 gig into the file and then gets timeouts.
@cef: Naw, security updates and version jumps don’t mix well… unless it’s Mozilla, of course! Haw haw.
Any chance of a backport for Gutzy?
jdub how come you were getting only these -
gutsy: 183824384 bytes transferred in 559 seconds (320.9K/s)
hardy: 183824384 bytes transferred in 145 seconds (1.21M/s)
-amounts.
on a lan…..
I will do my own tests and see what i get. I expect higher amounts using an older version of ssh on my network.
@dbmoodb: On a LAN? No way dude.
Er, as in, there is no way those numbers have anything to do with LAN speeds.
Ah you mean those are the speeds you can obtain over an internet connection ? - confused. I mean that ssh transfering at speeds of 1.21M/s seems a bit low on a lan if you mean over an internet connection that is a good speed up from an ok one.