The OpenSSH Bug That Wasn't

Much has been written about a purported OpenSSH vulnerability. On closer inspection, the reports actually got most of their facts wrong. Read on for the full story.

It all started with a blog post dated July 16, 2015, titled OpenSSH keyboard-interactive authentication brute force vulnerability (MaxAuthTries bypass), where the TL;DR is that it's possible to get an almost infinite number of tries at authentication -- good for bruteforce password guessing, for example -- if you only tickle the OpenSSH server just so.

Note: This piece is also available with trackers but nicer formatting here.

This sounded interesting and scary enough that I wanted to try it out myself. The blog quite helpfully supplies a one-liner that you can cut and paste to your own conmand line to check whether the systems you have within reach are indeed vulnerable.

Here's a transcript of running those tests on the machines I happened to try (Disclaimer: The recorded sessions here are from a second try, a few days after the first). First, my home gateway, running a recent OpenBSD 5.8-beta:

[Fri Jul 24 14:58:31] peter@elke:~$ ssh -lrazz -oKbdInteractiveDevices=`perl -e 'print "pam," x 10000'` skapet.bsdly.net
Host key fingerprint is SHA256:maeVFpNMibnkcwPSmjV4QBXfz5J97XLta6e2CrzsAYQ
+---[ECDSA 256]---+
|     .o=o+..     |
|      o.X+.o     |
|      EO.+* .    |
|      o.+oo+ =  .|
|        So=.o + o|
|         B   . o.|
|        . +   . +|
|         . +  .=.|
|         .+ .o+++|
+----[SHA256]-----+

razz@skapet.bsdly.net's password: 
Permission denied, please try again.
razz@skapet.bsdly.net's password: 
Permission denied, please try again.
razz@skapet.bsdly.net's password: 
Permission denied (publickey,password,keyboard-interactive).
[Fri Jul 24 16:53:06] peter@elke:~$

Here, razz is a non-existent user, and as we can see we get exactly three password prompts before the connection is shut down. Now we know that an essentially untweaked SSH server configuration on a recent OpenBSD does not behave as described in the article.

But what about earlier OpenBSD releases? I have one box I should have upgraded a while ago (to my enduring shame, it's still on 5.3-stable, but don't tell anyone). So here's the same thing pointed at that box:

[Fri Jul 24 16:53:06] peter@elke:~$ ssh -lrazz -oKbdInteractiveDevices=`perl -e 'print "pam," x 10000'` delilah         
Host key fingerprint is SHA256:AO8rn6Va9+b3+7gdVUxby5zWQFaLnkIA6wcEsOVHukA
+---[ECDSA 256]---+
|   Eoo.+..   .+.+|
|  . +o+ . .  .++B|
|   o oo+   . . O+|
|    ..+.. . . o .|
|     ...S. . o  .|
|       ..   .   .|
|    . o o      . |
|     + = .. .  o.|
|    ..+  oo. .=+o|
+----[SHA256]-----+

razz@delilah's password: 
Permission denied, please try again.
razz@delilah's password: 
Permission denied, please try again.
razz@delilah's password: 
Permission denied (publickey,password,keyboard-interactive).
[Fri Jul 24 16:59:37] peter@elke:~$

razz is not a valid user here either, and even the old OpenSSH version here shuts down the connection after three failures. I don't have any OpenBSD boxes with older versions than this anywhere I know about, and we can be reasonably confident that at least close to default configurations on OpenBSD are not vulnerable.

But several of the articles hint that OpenSSH on Linux is vulnerable. I do have a few CentOS boxes within reach. I'll repeat the test on one of these, then:

[Fri Jul 24 17:05:13] peter@elke:~$ ssh -lrazz -oKbdInteractiveDevices=`perl -e 'print "pam," x 10000'` nms
Host key fingerprint is SHA256:fdFxpvSDLq3W9D1d8U6RzuYQcd0WzAmIFfJAzcIkD8I
+---[RSA 2048]----+
|    .. o+==ooo*oB|
|     E. ++++ o+X=|
|         ....oo*.|
|         .  o.+ =|
|        S ...= ++|
|           .= =o+|
|           o . ++|
|          .     .|
|                 |
+----[SHA256]-----+

razz@nms's password: 
Permission denied, please try again.
razz@nms's password: 
Permission denied, please try again.
razz@nms's password: 
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

nms is a CentOS 6.6 box, and the result is as we can see here, pretty much identical to what we saw on the OpenBSD machines. So far we haven't seen anything like what the blogger kingcope had us to expect to see.

But looking back to the original article, he seems to have tested only on FreeBSD. I have some FreeBSD boxes within reach too, and rosalita runs a recent FreeBSD 10.1, freshly upgraded via freebsd-update. Here's what our experiment looks like pointed at rosalita:

[Fri Jul 24 17:15:03] peter@elke:~$ ssh -lrazz -oKbdInteractiveDevices=`perl -e 'print "pam," x 10000'` rosalita           
Host key fingerprint is SHA256:Ig6F8Au3f0KYNrzuc5qRrpZgY4Q/tz0bJrS0NZMxp1g
+---[ECDSA 256]---+
|.                |
| o .             |
|o + . E .        |
|.= * o *         |
|..X * B S        |
|.=o@.= +         |
|+ *oBo+          |
| =.oo=o.         |
|oo*+  .o         |
+----[SHA256]-----+

Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
Password for razz@rosalita:
^C

Bingo! We have finally seen the reported vulnerability in action on a live system. The ^C at the end is me pressing Ctrl-C to abort after ten tries, my heart wasn't quite in it for the full ten thousand. So far, it looks like this behavior is specific to FreeBSD, but of course it is conceivable that other systems ship with their sshds configured in a similar way.

After a bit of back and forth and reading articles elsehere, it seems that only OpenSSH servers that are set up to use PAM for authentication and with a very specific (non-default on OpenBSD and most other places) setup are in fact vulnerable. Even though there is a patch available which tightens up the code a bit in the PAM-specific parts, OpenBSD users don't actually need to apply it. One big reason being that OpenBSD does not use PAM for its authentication.

The question also came up in a thread on OpenBSD-misc, titled Alleged OpenSSH bug, where several OpenBSD developers commented. Do read the whole thread, but as we've already seen, it's easy to test whether your systems behave as described in the original blog post as well as this one.

And as OpenBSD developer Marc Espie says in his message,

Not surprisingly, as the patch clearly shows, the problem is right smack in the middle of USE_PAM code.
I wouldn't call that an OpenSSH bug. I would call it a systemic design flaw in PAM. As usual. LOTS of security holes in authentication systems stem from PAM. Why ? Because that stuff is over designed. Difficult to configure. Gives you MORE than you need to hang yourself several times over. It's been that way for as long as I can remember.

As they say, do read the whole thing. TL;DR this time around is: OpenBSD is not vulnerable, and on the systems that are, changing the configuration to close this particular bruteforcing opportunity is trivial. As is checking the facts before writing up a story. (And as several correspondents have reminded me already -- switching your sshd to keys only authentication will let you sleep better at night.)