How to deal with the recent random number generator bug in OpenSSL and update your SSH keys.

Edit: 16 may 2008: Please read this correction, if you’ve already read this article.

Edit: 19 may 2008: Doh! Also forgot to mention that you of course need to remove your vulnerable keys from any systems you uploaded them to. You can do this by removing them from your ~/.ssh/id_rsa file (see this comment for more info). Guess that teaches me about getting posts out in a hurry. Luckily, most upgraded SSH servers currently detect most compromised keys, so your compromised keys will usually be rejected.

You may have heard of the predictable random number generator bug in Debian and distributions based on Debian (like Ubuntu).

Those of you using SSH to log in to other systems are probably mostly affected by this, so here is a quick tutorial to get you back up and running:

First, make sure you upgrade your system. This is a prerequisite before fixing your SSH keys. On most Debian-based systems, your update process will be like this:

apt-get update
apt-get upgrade

You might need to add sudo before both commands if you’re not root and are on a system that allows you to sudo to root.

Presumably, your system will list that some SSL packages will need to be updated, so run the upgrade.

Your system might ask you to restart some processes, and recommend that you reboot. I’d recommend following up to this advice and just restarting your system, but if you really can’t, make sure you at least list the processes that make use of OpenSSL on your system to be restarted.

Next, depending on if you’re using Debian stable or unstable, or another distribution like Ubuntu, you might have a new application called ssh-vulnkey.

Simply running it without any arguments will try to determine if your SSH key is vulnerable, if you have any. Note that this might yield false negatives, so it’s best to replace your SSH keys anyway, but if you want to take the risk, that’s fine with me.

The output will be somewhat like this:

Unknown (no blacklist information): 2048 d0:c6:da:f5:e2:30:b0:3a:20:df:97:5a:47:2d:87:f0 /home/niek/.ssh/id_rsa.pub

Correction (16 may 2008): your output should look like this (this is important, because else the blacklist checker doesn’t have a good blacklist for your key):

Not blacklisted: 2048 d0:c6:da:f5:e2:30:b0:3a:20:df:97:5a:47:2d:87:f0 niek@lithium

You might need to install openssh-blacklist for it to show up like that, which you can do like this:

apt-get install openssh-blacklist

This presumably means that your key is not vulnerable. You still might want to generate a new one anyway (better safe than sorry, right?)

If you choose to generate a new SSH key, then first back up your old one:

mv .ssh .ssh.bak

Next, we can safely generate a new SSH key, just run:

ssh-keygen

and follow the interactive prompt.

Next, you might need to tell your servers about your new SSH key, but you just made a new one! Luckily you backed up your old key though, so you can still log in to your server like this:

ssh -i /location/to/the/backup/of/your/keyfile/here (presumably ~/.ssh.bak/id_rsa) yourserver.invalid

Next, put your new public key (which probably is in ~/.ssh/id_rsa.pub on your home system) in ~/.ssh/authorized_keys on your server, and log out. Now try to log in again with your new SSH key.

The host key(s) of your OpenSSH server might also be vulnerable, you can check this by issuing the following commands (obviously replacing yourserver.invalid with your actual SSH server):

ssh-keyscan -t rsa yourserver.invalid|ssh-vulnkey -
ssh-keyscan -t dsa yourserver.invalid|ssh-vulnkey -
ssh-keyscan yourserver.invalid|ssh-vulnkey -

Note that some of these might not work, depending on if your server has a host key with the algorithm we’re checking for.

The Gentoo wiki has a howto on how to generate host keys should you find yours vulnerable, or decide to update them anyway. Remember to restart your SSH server after updating those, and remember that you will probably get a complaint from your SSH client next time you try to login to your remote system that the host keys have changed.

Disclaimer: please note that I’m not ( currently at least ;-) ) a security or cryptography expert in any way, and some of my advice may be slightly incorrect or just wrong. This information is provided as is, without any warranty. Please contact a real security expert if you feel more secure with that, and don’t blame me if something goes wrong. Feel free to submit corrections in comments, and I’ll try to correct any blatant factual errors.

6 Responses to “How to deal with the recent random number generator bug in OpenSSL and update your SSH keys.”


  1. 1 Tchorix

    Hi Niek,

    Thanks for your post, but I have an observation… I tried yesterday the ssh-vulnkey command yesterday, and I got a message like yours, starting with “Unknown (no blacklist information):”.

    Today I installed the package openssh-blacklist (I had already openssl-blacklist yesterday (ssl instead of ssh)) and I got a different message:

    Not blacklisted: 2048 d0:c6:da:f5:e2:30:b0:3a:20:df:97:5a:47:2d:87:f0 /home/niek/.ssh/id_rsa.pub

    I replaced my key for the one you published on your post… To me, this message looks clearer, becuase it states that the key is “not blacklist”… the previous message looked more like it couldn’t determined if key was blacklisted or not…

    I hope it helps
    cheers
    Tchorix \m/

  2. 2 Niek Bergman

    @Tchorix:

    Thanks for notifying me about this.

    I figured out what’s causing this different output format. Turns out that it’s caused by a incomplete/missing blacklist for the current key type and bit amount. That you’re getting “Not blacklisted:” now, means that the key is probably indeed not vulnerable. It’s still good if you replace it though, if it was generated on a vulnerable Debian(-based) system.. you can never be sure enough ;-)

    You’ll probably indeed need to install openssh-blacklist for this, as those contain the blacklists. That package apparently also wasn’t installed on my home testing system, but it was on my server. Sorry for my misinformation there. I’ll immediately correct it in the blog entry.

    Also, I’ve noticed that OpenSSH will now usually reject logins by vulnerable keys, so you might have some users complaining that they’re not able to login if you manage a multi-user server, or you might lock yourself out if you update and try to login again using a vulnerable key.

  3. 3 Tchorix

    Hi Niek,

    I noticed about the output difference when I decided the install sshd on my machine, and I realized about the openssh-blacklist package. So this is consistent with what you mentioned (the difference between home and server).

    I believe my ssh-keys where generated with red-hat based systems a couple of years ago, but it wouldn’t hurt getting new keys anyway.

    cheers
    Tchorix

  4. 4 Grugnog

    Thanks for this post - it looks good generally, but misses one piece of very critical information. If you have (or suspect you may have) a vulnerable key then you must remove the public key from the authorized_keys file for *every server* or user account which you have added it to.

    Your post only mentions removing the key locally (which only prevents people accessing the ‘local’ machine) and adding the new key to servers. However, if you don’t remove the compromised key from each server that you have ever added it to, then all these servers themselves can be attacked using this vulnerability.

    I have a post up at http://www.civicactions.com/blog/howto_secure_your_ssh_ssl_and_openvpn_keys_generated_on_debian_ubuntu_and_related_distributions which folks might want to look at to cross check.

  5. 5 Grugnog

    For reference, here is a quote from the Ubuntu advisory:

    http://www.ubuntu.com/usn/usn-612-2
    > Once the user keys have been regenerated, the relevant public keys must be propagated to any authorized_keys files on remote systems. Be sure to delete the affected key.

  6. 6 Niek Bergman

    @Grugnog:

    Doh! I knew I missed something!

    Thanks very much.

Leave a Reply