So, you’re using a SSH key instead of a password. And your SSH key is protected by a passphrase, right? If so, you’re doing awesome! But there’s still one way a person can get around that:

  1. They gain access to your SSH key.
  2. Using a keylogger (or just watching you type), they get the passphrase protecting the SSH key.

With that, they could gain access to your remote servers.

How to protect it? Simple. Use “Multifactor” or Two Factor" Authentication (MFA / 2FA) to use another device to provide a temporary code needed to login.

And if you’re logging in a lot, such as copying lots of files, I’ll show you how not to have to enter that code each time. It’ll save you a lot of headache.

Warning

Use at your own risk. While doing this can increase the security of your system, if not configured and tested correctly it could allow unintended remote access to your system. System configurations change with time, and what follows could become out of date. After doing these steps, make sure to test and verify that it is working properly and did not create a way for the authentication to be bypassed.

The Plan

In order to do this, we will need to:

  1. Install a PAM module on the server, called “Google Authenticator” that comes with most Linux distros.
  2. Install a Multifactor Authentication App on your phone, if not already installed.
  3. Configure the server to allow KbdInteractiveAuthentication.
  4. Add a unique entry to your Multifactor Authentication App that you will use when you login via SSH.

Note

Note that ChallengeResponseAuthentication is a deprecated alias of KbdInteractiveAuthentication.

Note

These tools do not rely on Google services. They use mathematics to generate the code, without any need for the internet, making this setup vendor neutral.

Installing the Tools

Install the PAM module on the server and Multifactor Authentication App on your phone.

Install the PAM Module on the Server

Then on the server, install google-authenticator, use your package manager to install it.

sudo apt-get install libpam-google-authenticator
sudo dnf install libpam-google-authenticator
sudo pacman -Sy libpam-google-authenticator

Install a Multifactor app on your Phone (if needed)

If you already have a Multifactor app on your phone, you can skip this step.

Otherwise, open your phone’s store and search one. Some app options are:

  • Aegis
  • Google Authenticator
  • KeepassDX

Once you found the one you want, install it.

park bench overlooking the lake under autumn leaves

Fall day in Sandvika, Norway - credit

Setup the User’s MFA Tokens on the Server

On the server, make sure you are logged in as the user you login to via SSH.

Run the authenticator.

google-authenticator

It will prompt you on how you want to set it up.

  1. Yes, we want time based tokens.
Do you want authentication tokens to be time-based (y/n) y
  1. Here we can choose to only accept one code per 30 seconds. Answering y to this can prevent man-in-the-middle attacks, so a good idea even if you have bruteforce protection on your SSH port.
Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y
  1. And this last option will determine if we go with the default 1.5 minute time to enter the codes (n), or if we want a longer 4 minute one (y). Unless you are having issues with the entry of your codes, or have time issues on your server or phone, then going with the default here of n is good.
By default, a new token is generated every 30 seconds by the mobile app.
In order to compensate for possible time-skew between the client and the server,
we allow an extra token before and after the current time. This allows for a
time skew of up to 30 seconds between the authentication server and client. Suppose you
experience problems with poor time synchronization. In that case, you can increase the window
from its default size of 3 permitted codes (one previous code, the current
code, the next code) to 17 permitted codes (the eight previous codes, the current
code, and the eight next codes). This will permit a time skew of up to 4 minutes
between client and server.
Do you want to do so? (y/n) n
  1. You’ll be provided with a QR code.

Tip

For those that want an automated way to do this:

google-authenticator -t -d -f -r 3 -R 30 -w 3

Add to Phone’s Multifactor Authentication App

Scan the QA code with your Multifactor Authentication App. Or you may enter the provided code into the App.

Once done, your app will provide you with a code. Enter this back into the above prompt to confirm it is working as expected.

Now we need to have the server require it.

Tip

Backup your key and recovery codes found in ~/.google-authenticator to a safe location. Otherwise you will be locked out if you loose your MFA device.

Scenic river and bridge

Scenic river and bridge over Wissahickon Valley Park, Philadelphia, US - credit

Configuring it on the Server

SSH Challenge Response on the Server

Open up a different terminal and login into your server.

Make a backup copy of your SSH configuration

sudo cp /etc/ssh/sshd_config{,.bak}

Login to your server to establish a second connection.

Warning

Be logged into the server with at least two connections/terminals. And have a backup non-SSH method of logging into your server, in case you are completely disconnected.

We will restart SSH, and even though we’ll show you how to check the configuration to minimize the risk of being disconnected, there is still will some risk of needing another way to log back in.

We need to enable KbdInteractiveAuthentication in the SSH configuration.

Open /etc/ssh/sshd-config in your editor, and find the line with this setting. set it to yes. Or you may need to add/uncomment it.

KbdInteractiveAuthentication yes

Info

Do not put KbdInteractiveAuthentication in the Match section, as it’s not supported there.

Check that UsePAM is enabled.

UsePAM yes

And we only want to use our Keyfile and MFA code, so add/set this.

AuthenticationMethods publickey,keyboard-interactive

Verify your SSH configuration.

sudo sshd -t ; echo $?

This will print an error if there’s an issue.

If it doesn’t print anything and the exit code is 0, then restart SSH:

sudo systemctl restart sshd

In a third terminal window, verify that you can still login like you normally do. We haven’t enabled the Multifactor Authentication, so you won’t need that yet.

Enabling MFA with PAM

You now have the authenticator setup. But we need to add it to your server’s login configuration, /etc/pam.d/sshd, so PAM will know to use it. This will only be for the server’s SSH logins. Console and other logins will not be effected.

First let’s make a backup.

sudo cp -a /etc/pam.d/sshd{,-bak}

Open it up in an editor

sudo nano /etc/pam.d/sshd
sudo vim /etc/pam.d/sshd

Add this to the end of /etc/pam.d/sshd.

auth required pam_google_authenticator.so nullok
auth required pam_permit.so

If you use keyfile authentication with your server, then comment this out. Otherwise you’ll be asked for your password along with your keyfile (so, 3 forms of authentication, if you want it.)

# Standard Un*x authentication.
#@include common-auth

Info

Don’t use this if you use password authentication, as it disables it and you will be locked out.

Save it.

bridge in middle of jungle

Almost there… a swing bridge in New Zealand - credit

Test it out

And (you know what’s coming) in a different terminal on your local machine, login to your server.

ssh ExampleHost

It should now require your MFA code from your phone. Enter it when prompted.

debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
Authenticated using "publickey" with partial success.
debug1: Authentications that can continue: keyboard-interactive
debug1: Next authentication method: keyboard-interactive
(YourUser@ExampleHost) Verification code:                 <-- Enter your code here, it will not echo.
Authenticated to ExampleHost using "keyboard-interactive".
debug1: channel 0: new [client-session]
debug1: Entering interactive session.

If it is not prompting for your MFA code, then check the permissions of the MFA file on the server.

ls -lh ~/.google_authenticator

That should show something like

-r-------- 1 YourUser YourUser 100 May  1 18:16 /home/YourUser/.google_authenticator

If the permissions aren’t what you see above, change them:

chmod 600 ~/.google_authenticator

Three Factor Authentication

You can set it to require a password, a keyfile, and a MFA code each time you login.

To do so, we previously commented out #@include common-auth. And AuthenticationMethods only required two methods. We’ll need to change both of those.

Open /etc/ssh/sshd-config and set AuthenticationMethods to this:

AuthenticationMethods publickey,password publickey,keyboard-interactive

Open /etc/pam.d/sshd and uncomment/re-enable #@include common-auth:

# Standard Un*x authentication.
@include common-auth
auth required pam_google_authenticator.so
auth required pam_permit.so

Be sure to test ssh before restarting.

sudo sshd -t && sudo systemctl restart sshd

Require MFA for Everyone

Now, as you noted before, we set the MFA to be optional, only if the user had it setup. We can make it required for all users.

Warning

If enabled, any users that do not have MFA setup will be locked out. New users will need a method to setup MFA up for their account, such as when they setup their password, before they will be able to login.

To do so, remove nullok from the two lines we added to the bottom of /etc/pam.d/sshd so the two lines look like:

auth required pam_google_authenticator.so
auth required pam_permit.so

Save it and test it out again. No service restarted is needed.

Double check

Be sure to check for both that you are able to login using MFA, and you are not able to login in undesired ways.

Warning

Make sure to test it out thoroughly, including without a password/key, and with a different user. If there is a configuration issue on the server, it could allow remote access to your system.

For fun, you can even try it with your phone on airplane mode. Just to verify that the MFA app doesn’t require an internet connection.

Avoiding MFA Exhaustion

In some cases using MFA will be a royal pain, needing to enter that dang code each and every time.

My post Too Many Multifactor Codes: Sane SSH Logins with ControlMaster covers how to avoid this, after the initial login, by using the ControlMaster function of SSH to automatically reuse a single open connection.

The initial login still has all the multifactor authentication that you’ve set up. The change is that it reuses the initial SSH’s connection so it can use it again. Think of it like putting everything down the same pipe, rather than making a new one each time.

Thus you can use rsync, scp, and other one time commands without having to enter that pesky code each and every time.

Conclusion

With that, your server is more secure.

References