sshca—tool to manage SSH CA certificates

Lars Wirzenius

2022-09-06 06:13

1 Introduction

The sshca tool helps manage an SSH Certificate Authority (CA) and create host and user certificates. Such certificates make using and administering SSH less tedious and more secure, by removing the need for users to check host keys, or maintain authorized_keys files.

An SSH CA is an SSH key dedicated to signing, or certifying, other SSH keys. Such a signed key is called a certificate and is used together with the private part of the certified key. The certificate is used instead of the public key.

SSH clients and servers can be configured to trust certificates made by one or more CA keys. This makes it possible for a client to trust a server without asking the user to accept the host key for each new server. A server can trust a client without having the client’s public key configured for that user in the authorized_key file. This simplifies overall key management significantly, but requires creating and managing CA keys and certificates.

1.1 Host certificates

Traditionally, in the world of SSH, servers have host keys that rarely change, but are generated separately for each host. When a user accesses a host for the first time, at a given address, they are presented with the host’s public key, and need to manually, laboriously, and usually insecurely, check that it’s the right key for that host.

This can be a risky situation: if an attacker manages to trick the user’s SSH client to show a key the attacker has generated, and the user accepts it as the real one, the attacker can see – and change – all the traffic going over the SSH connection. This mostly nullifies the security benefit SSH is meant to provide. (Not entirely: the connection is still protected against other attackers.)

In a situation where there are many hosts, or hosts gets recreated often, or change address a lot, all of which happen when using cloud technologies, the risky situation keeps happening frequently. Not only is it risky, it is also tedious and cumbersome to the user. If this keeps happening a lot, users are in effect trained to automatically accept all host keys. This is an example of bad usability being bad security.

The risky situation can be avoided by having the host keys be communicated to all users ahead of time, but doing this in a secure and convenient way is difficult. It is also unnecessary.

Using an SSH CA to certify SSH host keys means the user’s SSH client can trust it without asking the user to verify it. The client is configured to trust any host certificate that can be verified using the SSH CA public key. The CA public key still needs to be communicated to the user in a secure way, but the CA key is only one key and rarely changes, so the tiresome risky situation happens very rarely. After the user has the CA key, an attacker can’t trick the user into accepting a false host key.

With host certificates, the SSH client never needs to ask its user if the host key of a new host is valid, and the user never needs to try to verify it. If the host’s identity (host key or address) changes, such as when a virtual machine is re-created, the client doesn’t need to bother the user about it, as long as the new identity gets a new certificate.

Overall, this leads to a much smoother and more secure experience for people using SSH.

1.2 User certificates

Traditionally, a user authenticates themselves to an SSH server using a password, or a user key. We will not discuss passwords here. You should not use passwords. Your SSH server should not accept passwords.

A user SSH key is an SSH key pair of which the user has the private part. The public part is added to the ~/.ssh/authorized_keys file on the server for the user’s account. At login time, the client proves to the server that it has the private key that corresponds to one of the public keys in that file, and this proves to the server that the user is authorized to log in. This is great, because the user does not need to remember a strong password, nor type it in every time they log in, and the server does not need to store the user’s password at all, even in an encrypted way.

The result is an easy, secure way for the user to log into the server. However, this only works if the list of authorized keys is kept up to date.

If the user needs to, for any reason, change to a new key, perhaps as part of a regular key rotation strategy, the list of authorized keys needs to be updated to add the new key, and remove the old key. This needs to be done on each server the user uses. The update procedure is often a risky, tedious step. If an attacker manages to get the attacker’s key into the list, they can log into the server as the user. Given that the authorized_keys file is usually user-editable, the user may add any SSH public keys to that file, including keys for other people, or keys stored on machines that are insecure. The user may do this intentionally, or because they’ve been tricked or coerced into doing it.

An SSH CA can create a user certificate, which ties a user’s SSH public key to a username. An SSH server can be configured to trust such certificates, made with specific CA keys, and to act as if that user’s public key is in their authorized_keys file, even if that file doesn’t exist. The result is that there is no need to maintain that file. This also means it’s feasible to revoke access with specific certificates.

The user certificate replaces the public key in the SSH authentication process. The user still needs the corresponding private key to authenticate: the certificate itself is not enough.

Overall, this leads to system administrators having an easier way to control who has access their servers over SSH.

1.3 Certificate automation

Generating all these certificates can be done using the ssh-keygen command line tool. However, it’s just intricate enough that it becomes tedious and cumbersome and thus error prone. The sshca tool makes it easier.

1.4 Configuring servers and user accounts

The sshca tool does not install host certificates on servers, nor configure servers or user accounts to trust certificates made using specific CA identities. The server system administrators and users need to do that themselves.

1.5 SSH CA vs SSHFP

Another approach is to distribute host keys via DNSSEC using SSHFP DNS records. This requires DNSSEC to work for all clients, and only works for verifying host identities, not user identities. However, they may be easier to adopt for some organizations.

2 The sshca command line tool

The sshca tool maintains a secure storage of CA key pairs, and host and user public keys, and can use the CA keys and the stored public keys to generate host and user certificates.

2.1 The store

Security note: The sshca tool maintains a store of SSH public and private keys, as a directory on the local file system. This store is assumed to be trusted: any key there is assumed to have been vetted before being added. The user of the tool should ensure the store can only be accessed by them and not by other parties. The security and integrity of the SSH CA system maintained by sshca depends on that.

The store is kept in ~/.local/state/sshca by default, but the location can be configured via the tool configuration file.

The way the store is implemented should probably be improved.

2.2 sshca built-in help

See the sshca help command for a list of its subcommands and how to invoke everything. The --help option is an alias.

To avoid repetition and to avoid getting out of sync, this document doesn’t repeat all the commands and options.

2.3 Revocations

The sshca tool does not support revoking certificates. Revocations can be done manually using ssh-keygen, but it may be easier to use certificates with short validity periods and creating and distributing new certificates when needed. This is easier for host certificates, which are under direct system administrator control.

For user certificates, a self-serve system would be good, but not currently available. However, the system administrator can generate and publish new user certificates frequently. User certificates are not secret, and they’re tightly tied to the user’s private key. User certificates are useless without the private key.

2.4 Tool configuration

In ~/.config/sshca/config.yaml (or other location as specified according to the XDG directory standard), a configuration file can specify:

3 Requirements for SSH CA automation

Tooling to automate SSH CA management needs to satisfy all the following high-level requirements to be acceptable.

The following sections document more detailed acceptance criteria and how they are verified in an automated manner.

3.1 Smoke test

This scenario verifies that the sshca command line tool can be invoked at all, in the simplest possible ways.

given an installed sshca
when I run sshca --help
then stdout contains "--help"

3.2 Configuration lookup

This scenario verifies that the sshca command line tool can show its actual configuration.

given an installed sshca
when I run sshca config
then stdout contains "store"

3.3 CA key management

It must be possible to manage multiple SSH CA keys. This scenario verifies that sshca can create and remove CA keys.

Initially the store must be empty and have no CA keys.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca list
then stdout is exactly ""
when I run sshca ca list --json
then stdout contains "[]"

File: config.yaml

store: store.yaml

When we create two new CA keys, for host and user certification, they both show up in the list.

when I run sshca ca new user userCAv1
and I run sshca ca show userCAv1
then stdout contains "userCAv1"
and stdout contains "ssh-ed25519"
when I run sshca ca new host hostCAv1
and I run sshca ca show hostCAv1
then stdout contains "hostCAv1"
and stdout contains "ssh-ed25519"
when I run sshca ca list
then stdout is exactly "hostCAv1\nuserCAv1\n"
when I run sshca ca list --json
then stdout contains "hostCAv1"
and stdout doesn't contain "-----BEGIN OPENSSH PRIVATE KEY-----"

We can see the CA public key.

when I run sshca ca public-key hostCAv1
then stdout matches regex ^ssh-ed25519\s\S+\s$
when I run sshca ca public-key userCAv1
then stdout matches regex ^ssh-ed25519\s\S+\s$

When we remove a CA key, it’s no longer in the store.

when I run sshca ca remove hostCAv1
and I run sshca ca list
then stdout contains "userCAv1"
when I run sshca ca remove userCAv1
and I run sshca ca list
then stdout is exactly ""

When we create two CA keys, they can be individually removed.

when I run sshca ca new host CAv1
and I run sshca ca new host CAv2
and I run sshca ca list
then stdout contains "CAv1"
and stdout contains "CAv2"
when I run sshca ca remove CAv1
and I run sshca ca list
then stdout doesn't contain "CAv1"
and stdout contains "CAv2"
when I run sshca ca remove CAv2
and I run sshca ca list
then stdout is exactly ""

3.3.1 Rename CA in store

Requirement: We must be able to renamme a CA in the store.

Justification: Sometimes it’s necessary to change the name of a CA, and it’s convenient to be able to just rename them without having to export and then re-import their data.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new user sillyca
and I run sshca ca rename sillyca userCAv1
and I run sshca ca list
then stdout contains "userCAv1"
and stdout doesn't contain "sillyca"

3.4 Host management

The sshca tool needs to manage some information about hosts. At minimum, the host’s name and public key. However, sshca can also generate a host key, and store the private key as well.

3.4.1 Import host public key into store

Requirement: It must be possible to import a host’s public key into the store.

First we verify there are no hosts in the store.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca host list
then stdout is exactly ""
when I run sshca host list --json
then stdout contains "[]"

Then we create a host key pair and import the public key into the store.

when I run ssh-keygen -t ed25519 -N '' -f myhost
and I run sshca host new myhost.example.com myhost.pub
and I run sshca host show myhost.example.com
then stdout contains "myhost.example.com"
and stdout contains "ssh-ed25519"
when I run sshca host list
then stdout contains "myhost.example.com"
when I run sshca host list --json
then stdout contains "myhost.example.com"
and stdout doesn't contain "-----BEGIN OPENSSH PRIVATE KEY-----"

We must not be able to import another public key for the same host.

when I run ssh-keygen -f myhost2 -N ""
and I try to run sshca host new myhost.example.com myhost2.pub
then command fails
and stderr contains "myhost.example.com"
when I run sshca host list
then stdout contains "myhost.example.com"

3.4.2 Add principals to a new host

Requirement: We must be able to set any number of principals for a host.

Justification: Many hosts have multiple name, such as foo and foo.example.com.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca host generate myhost --principal alfred.lan -p alfred.example.com
and I run sshca host principals list myhost
then stdout doesn't contain "myhost"
and stdout contains "alfred.lan"
and stdout contains "alfred.example.com"

3.4.3 Manage principals of an existing host

Requirement: It must be possible to change principals to an existing host.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca host generate myhost
and I run sshca host principals list myhost
then stdout doesn't contain "alfred.lan"
and stdout doesn't contain "alfred.example.com"
when I run sshca host principals add --host myhost alfred.lan alfred.example.com
and I run sshca host principals list myhost
then stdout contains "alfred.lan"
and stdout contains "alfred.example.com"

3.4.4 Rename host in store

Requirement: We must be able to renamme a host in the store.

Justification: Sometimes hosts change name, and it’s convenient to be able to just rename them without having to export and then re-import their data.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca host list
then stdout is exactly ""
when I run sshca host generate myhost
and I run sshca host rename myhost newhost
and I run sshca host list
then stdout contains "newhost"
and stdout doesn't contain "myhost"

3.4.5 Remove host from store

Requirement: We must be able to remove a host from the store.

Justification: We don’t want the store to grow in size indefinitely.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca host list
then stdout is exactly ""
when I run ssh-keygen -t ed25519 -N '' -f myhost
and I run sshca host new myhost.example.com myhost.pub
and I run sshca host remove myhost.example.com
and I run sshca host list
then stdout is exactly ""

3.4.6 Generate a host key

Requirement: add a host by generating a host key.

Justification: this enables a use case where host keys are installed onto a system rather than generated on the system. This is useful, for example, when first installing a system: the installation environment may not be able to generate a good key, and certainly doesn’t have the CA private key to create a certificate.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca host generate myhost.example.com
and I run sshca host list
then stdout contains "myhost.example.com"
when I run sshca host public-key myhost.example.com
then stdout contains "ssh-ed25519 "
when I run sshca host private-key myhost.example.com
then stdout contains "-----BEGIN OPENSSH PRIVATE KEY-----"

3.4.7 Re-generate a host key

Requirement: we can generate a new host key for an existing host that already has a private key.

Justification: Sometimes it’s good to create a new host key. It’s convenient if one doesn’t need to remove the host first, and can keep all other data about the host. However, re-generating a key should only be possible for a host that has a private key (which means it was generated with the sshca tool).

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run ssh-keygen -t ed25519 -N '' -f myhost
and I run sshca host new myhost myhost.pub
and I try to run sshca host generate myhost
then command fails
when I run sshca host remove myhost
and I try to run sshca host regenerate myhost
then command fails
when I run sshca host generate myhost
and I run sshca host regenerate myhost
and I run sshca host list
then stdout contains "myhost"
when I run sshca host public-key myhost
then stdout contains "ssh-ed25519 "
when I run sshca host private-key myhost
then stdout contains "-----BEGIN OPENSSH PRIVATE KEY-----"

3.4.8 Export host public and private keys

Requirement: It must be possible to export a host’s public key, and also the private key when it’s known.

Justification: Exporting the public key is handy for debugging. Exporting the private key enables sshca to be the canonical source of truth for a host’s SSH identity, which is handy for setting the identity via configuration management, and for initial system installation before the first boot.

However, the private key can only be exported if sshca generated the key. If a host public key was imported, there is no private key stored for the host.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I try to run sshca host public-key myhost.example.com
then command fails
when I try to run sshca host private-key myhost.example.com
then command fails
when I run ssh-keygen -t ed25519 -N '' -f myhost
and I run sshca host new myhost.example.com myhost.pub
and I run sshca host public-key myhost.example.com
then stdout contains "ssh-ed25519 "
when I try to run sshca host private-key myhost.example.com
then command fails
when I run sshca host generate otherhost.example.com
and I run sshca host public-key otherhost.example.com
then stdout contains "ssh-ed25519 "
when I run sshca host private-key otherhost.example.com
then stdout contains "-----BEGIN OPENSSH PRIVATE KEY-----"

3.4.9 Certify a host

Requirement: we must be able to create a host certificate.

Justification: creating certificates is the reason for the tool to exist.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new host CAv1
and I run sshca host generate myhost.example.com
and I run sshca host certify CAv1 myhost.example.com
then stdout matches regex ^ssh-ed25519-cert-v01@openssh.com
when I run sshca host certify --output my.cert CAv1 myhost.example.com
then file my.cert matches regex /^ssh-ed25519-cert-v01@openssh.com/

3.4.10 By default host certificates are valid for 90 days

Requirement: By default, host certificates should be valid a limited time.

Justification: This is a bit of a gut feeling rather than a proper reason, but it doesn’t seem useful for a certificate to be valid forever, just like a TLS certificate from Let’s Encrypt isn’t.

Note that due to it being hard to parse, we just check that there is a validity period set in the certificate, rather than checking it’s 90 days.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new host CAv1
and I run sshca host generate myhost.example.com
and I run sshca host certify --output my.cert CAv1 myhost.example.com
and I run ssh-keygen -L -f my.cert
then stdout matches regex Valid: from \d+-\d+-\d+T\d+:\d+:\d+ to \d+-\d+-\d+T\d+:\d+:\d+

3.4.11 Host certificate validity can be set

Requirement: We can create a host certificate with a limited validity period.

Justification: Sometimes the default validity is too long, such as when doing an initial install on a system, when the host private key can’t be secured strongly. We can work around that by having a very short certificate lifetime for the initial install, and then install a new host private key and longer-lived certificate after the system has booted.

Note that due to it being hard to parse, we just check that there is a validity period set in the certificate, rather than checking it’s what we specify.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new host CAv1
and I run sshca host generate myhost.example.com
and I run sshca host certify --output my.cert CAv1 myhost.example.com --expires-in 1d
and I run ssh-keygen -L -f my.cert
then stdout matches regex Valid: from \d+-\d+-\d+T\d+:\d+:\d+ to \d+-\d+-\d+T\d+:\d+:\d+

3.5 User management

The sshca tool needs to manage some information about users: their public key and username.

3.5.1 Import user public key into store

Requirement: It must be possible to import a user’s public key into the store.

First we verify there are no users in the store.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca user list
then stdout is exactly ""
when I run sshca user list --json
then stdout contains "[]"

Then we create a user key pair and import the public key into the store.

when I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub
and I run sshca user show myname
then stdout contains "myname"
and stdout contains "ssh-ed25519"
and stdout doesn't contain "private_key"
when I run sshca user list
then stdout contains "myname"
when I run sshca user list --json
then stdout contains "myname"
and stdout doesn't contain "-----BEGIN OPENSSH PRIVATE KEY-----"

We must not be able to import another public key for the same user.

when I run ssh-keygen -N "" -f myself2
and I try to run sshca user new myname myself2.pub
then command fails
and stderr contains "myname"
when I run sshca user list
then stdout contains "myname"

3.5.2 Add principals to a new user

Requirement: It must be possible to set any number of principals for a new user.

Justification: the same user might need to log in as themselves, or into various role accounts, such as root, in the realm of hosts that trust a CA.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run ssh-keygen -t ed25519 -N '' -f tomjon
and I run sshca user new tomjon tomjon.pub --principal root -p king
and I run sshca user principals list tomjon
then stdout doesn't contain "tomjon"
and stdout contains "root"
and stdout contains "king"

3.5.3 Manage principals of an existing user

Requirement: It must be possible to change principals to an existing user.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run ssh-keygen -t ed25519 -N '' -f tomjon
and I run sshca user new tomjon tomjon.pub
and I run sshca user principals list tomjon
then stdout doesn't contain "root"
and stdout doesn't contain "king"
when I run sshca user principals add --user tomjon root king
and I run sshca user principals list tomjon
then stdout contains "root"
and stdout contains "king"
when I run sshca user principals remove --user tomjon king
and I run sshca user principals list tomjon
then stdout contains "root"
and stdout doesn't contain "king"

3.5.4 Rename user in store

Requirement: We must be able to rename a user in the store.

Justification: Sometimes the user changes their name and so the username changes. It’s convenient to be able to rename them, instead of exporting the public key, removing the user, and adding them back with a new username.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub
and I run sshca user rename myname newname
and I run sshca user list
then stdout contains "newname"
and stdout doesn't contain "myname"

3.5.5 Remove user from store

Requirement: We must be able to remove a user from the store.

Justification: We don’t want the store to grow in size indefinitely.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub
and I run sshca user remove myname
and I run sshca user list
then stdout is exactly ""

3.5.6 Export user public key

Requirement: It must be possible to export a host’s public key.

Justification: Exporting the public key is handy for debugging.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I try to run sshca user public-key myname
then command fails
when I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub
and I run sshca user public-key myname
then stdout contains "ssh-ed25519 "

3.5.7 Certify a user

Requirement: we must be able to create a user certificate.

Justification: creating certificates is the reason for the tool to exist.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new user CAv1
and I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub --principal tomjon -p king
and I run sshca user certify CAv1 myname
then stdout matches regex ^ssh-ed25519-cert-v01@openssh.com
when I run sshca user certify --output my.cert CAv1 myname
then file my.cert matches regex /^ssh-ed25519-cert-v01@openssh.com/
when I run ssh-keygen -Lf my.cert
then stdout doesn't match regex ^\s+myname$
and stdout matches regex ^\s+tomjon
and stdout matches regex ^\s+king

3.5.8 By default user certificates are valid for 90 days

Requirement: By default, user certificates should be valid a limited time.

Justification: We have not reliable way of revoking a certificate, or, rather, distributing the revocation to all hosts that may trust our CA, unless they are all under own control. However, creating a new certificate is easy and only needs to be distributed to to the user.

Note that due to it being hard to parse, we just check that there is a validity period set in the certificate, rather than checking it’s 90 days.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new user CAv1
and I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub
and I run sshca user certify --output my.cert CAv1 myname
and I run ssh-keygen -L -f my.cert
then stdout matches regex Valid: from \d+-\d+-\d+T\d+:\d+:\d+ to \d+-\d+-\d+T\d+:\d+:\d+

3.5.9 User certificate validity can be set

Requirement: We can create a user certificate with a limited validity period.

Justification: Sometimes the default validity is too long, such as when we want to limit the time window in which a compromised user private key may be used by an attacker.

Note that due to it being hard to parse, we just check that there is a validity period set in the certificate, rather than checking it’s what we specify.

given an installed sshca
and file .config/sshca/config.yaml from config.yaml
when I run sshca ca new user CAv1
and I run ssh-keygen -t ed25519 -N '' -f myself
and I run sshca user new myname myself.pub
and I run sshca user certify --output my.cert CAv1 myname --expires-in 1d
and I run ssh-keygen -L -f my.cert
then stdout matches regex Valid: from \d+-\d+-\d+T\d+:\d+:\d+ to \d+-\d+-\d+T\d+:\d+:\d+

4 SEE ALSO

5 Thanks

While writing this, the author got feedback and reviews of drafts from David Leggett.