Skip to content

SSH Certificates#

Oinit provides a set of tools to integrate ssh-certificates with federated identities, and uses Motley Cue as a backend. It provides helper tools at several places to improve the user experience and lower the barrier of entrance.

The particular difference of our solution from of-the-shelf ssh-certificates are:

  • oinit is based on OIDC access tokens
  • oinit contacts the Motley Cue instance identified for a given ssh host.
  • oinit-ca verifies tokens with Motley Cue
  • oinit-ca enforces the username obtained from Motley Cue in the ssh-certificate (by using “CriticalOptions: force-command oinit-switch <username>”)
  • oinit-switch is a helper-shell that ensures the correct user will be logged in.
  • oinit-sshd xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Login flow#

          SSH Certificate-based Authentication Flow with oinit
          ====================================================
  SSH Client              SSH Client              CA Server           SSH Server      
  User                    oinit                   oinit-ca            motley_cue
  ----------              ----------              ----------          ----------
   |  1. ssh user@host        |                        |                     |
   |------------------------->|                        |                     |
   |                          | 2. Get OIDC token      |                     |
   |                          |   from oidc-agent      |                     |
   |                          |                        |                     |
   |                          | 3. Send token +        |                     |
   |                          |    SSH public key      |                     |
   |                          |----------------------->|                     |
   |                          |                        | 4. POST /verify_user|
   |                          |                        |-------------------->|
   |                          |                        | 5. Validate token   |
   |                          |                        |    Return principals|
   |                          |                        |                     |
   |                          |                        | 6. User info        |
   |                          |                        |<--------------------|
   |                          |                        | 7. Sign SSH cert    |
   |                          |                        |    with CA key      |
   |                          | 8. Return SSH cert     |                     |
   |                          |<-----------------------|                     |
   |                          |                                              |
   |                          | 9. Add cert to SSH agent                     |
   |                          |                                              |
   |                          | 10. Connect to SSH server with cert          |
   |                          |--------------------------------------------->|
   |                                                 (trusts CA pubkey)      |
   |                                                                         |
   |  11. SSH session granted                                                |
   |<------------------------------------------------------------------------┘

Client Side#

Client side integration works with the tool oinit. It is used to configure which hosts support oinit enabled ssh-certificates, using oinit add <hostname>.

oinit is the client-side component of the oinit certificate management system. It handles the automatic retrieval of SSH certificates when connecting to SSH servers that are managed by an oinit Certificate Authority (CA). The client integrates seamlessly with OpenSSH by using the SSH ProxyCommand/Match exec mechanism.

How It Works#

  1. Host Registration: Users use oinit add to register SSH hosts that support certificate-based authentication
  2. SSH Integration: oinit modifies your SSH config, so it can request ssh-certificates for connections to registered hosts
  3. Certificate Retrieval: When you SSH to a registered host, oinit automatically:
  4. Retrieves an OIDC token from oidc-agent
  5. Requests a signed SSH certificate from the oinit-ca server
  6. Adds the certificate to your SSH agent
  7. Transparent Connection: SSH proceeds normally using the certificate for authentication

Installation#

Prerequisites#

  • oidc-agent must be installed and running for OIDC token management
  • ssh-agent must be running to store certificates
  • OpenSSH client

Setup#

  • Install the oinit binary (typically to /usr/bin/oinit)
apt install oinit

yum install oinit

zypper install oinit
  • Ensure oidc-agent is running with at least one configured account
  • Start ssh-agent if not already running

Configuration#

File Locations#

oinit uses the following configuration files:

  • ~/.ssh/oinit_hosts - User-specific list of managed hosts
  • /etc/ssh/ssh_oinit_hosts - System-wide list of managed hosts
  • ~/.ssh/config - Modified (by oinit) to include the oinit Match block
  • ~/.ssh/known_hosts - Stores CA public keys for host verification

SSH Config Integration#

When you add your first host, oinit automatically adds this block to your SSH config:

Match exec "oinit match %h %p"
    User oinit

This tells SSH to run oinit match before connecting. If the host is managed by oinit, it will obtain a certificate before the connection proceeds.

Environment Variables#

  • OIDC_AGENT_ACCOUNT - Pre-select a specific oidc-agent account
  • OIDC_ISS or OIDC_ISSUER - Pre-select an OIDC issuer
  • ACCESS_TOKEN, OIDC, OS_ACCESS_TOKEN, OIDC_ACCESS_TOKEN, WATTS_TOKEN, WATTSON_TOKEN - Provide token directly instead of using oidc-agent

Commands#

oinit add#

Add a host to be managed by oinit:

# Automatic CA discovery via DNS
oinit add login.example.com

# Specify port
oinit add login.example.com:2222

# Manual CA specification
oinit add login.example.com https://ca.example.com:8443

This command: - Determines the CA (from DNS or manual specification) - Contacts the CA to get its host public key - Adds the host to ~/.ssh/oinit_hosts - Adds the CA public key to ~/.ssh/known_hosts - Modifies ~/.ssh/config if needed

oinit delete#

Remove a host from oinit management:

oinit delete login.example.com
oinit delete login.example.com:2222

This removes the host from your configuration and clears any cached certificates from ssh-agent.

oinit list#

List all hosts managed by oinit:

oinit list

Shows both user-configured and system-wide configured hosts.

Basic Usage#

# Add a host
oinit add compute.university.edu

# SSH to the host - certificate is obtained automatically
ssh username@compute.university.edu

Multiple OIDC Providers#

When a host supports multiple OIDC providers, oinit will prompt you to choose:

[1] https://oidc.university.edu (Accounts: alice@uni.edu)
[2] https://login.federation.org
Please select a provider to use [1-2]: 1

Pre-selecting Provider#

# Using account name
export OIDC_AGENT_ACCOUNT=alice@uni.edu
ssh username@compute.university.edu

# Using issuer URL  
export OIDC_ISS=https://oidc.university.edu
ssh username@compute.university.edu

Non-Standard SSH Port#

# Add host with custom port
oinit add login.example.com:2222

# SSH normally - oinit handles the port
ssh -p 2222 username@login.example.com

Troubleshooting#

Common Issues#

  1. “oidc-agent is not running”

    • Start oidc-agent: eval $(oidc-agent)
    • Configure an account: oidc-add

  2. “ssh-agent is not running”

    • Start ssh-agent: eval $(ssh-agent)

  3. “Could not determine CA from DNS”

    • Manually specify the CA: oinit add hostname https://ca-server:port
    • Check with your administrator for the correct CA URL

  4. Certificate not being used

    • Verify the Match block exists in ~/.ssh/config
    • Check that the host is listed in oinit list
    • Ensure your SSH username matches the certificate principals

Security Considerations#

  • Certificates are generated with unique ED25519 keys for each request
  • Private keys exist only in memory and are never written to disk
  • Certificates include the host as a principal to prevent misuse
  • Certificate lifetime is limited by token validity

Server Side: The CA#

oinit-ca is the Certificate Authority (CA) server component of the oinit system. It acts as an SSH certificate authority that issues short-lived SSH certificates to users who authenticate with OpenID Connect (OIDC) tokens. The server integrates with motley_cue for user authentication and authorisation.

oinit-ca may be installed on an arbitrary host. For the ease of deployment, we support it to be installed on the SSH server host, just as Motley Cue.

Architecture#

The oinit-ca server:

  • Exposes a REST API for certificate operations
  • Maintains SSH CA key pairs (separate for host and user certificates)
  • Validates OIDC tokens through motley_cue
  • Issues time-limited SSH certificates
  • Supports multiple host groups with different CA keys and policies

Installation#

Prerequisites#

  • Access to a motley_cue instance for OIDC token validation
  • SSH CA key pairs (can be generated with ssh-keygen)

Packages#

apt install oinit-ca

yum install oinit-ca

zypper install oinit-ca

Docker#

make oinit-ca-docker
# Creates a Docker image tagged as oinit-ca

Configuration#

The server uses an INI-format configuration file with:

  • Global defaults section
  • Host group sections for different sets of hosts

Example configuration (/etc/oinit-ca/config.ini):

# Global defaults - can be overridden per host group
host-ca-privkey = /etc/oinit-ca/host-ca
host-ca-pubkey  = /etc/oinit-ca/host-ca.pub
user-ca-privkey = /etc/oinit-ca/user-ca
user-ca-pubkey  = /etc/oinit-ca/user-ca.pub

# Certificate validity: "token" or seconds (e.g., 3600 = 1 hour)
cert-validity = token
## Fixed certificate validity of 8 hours
# cert-validity = 28800

# How long to cache motley_cue responses (seconds)
cache-duration = 600

# Host group for example.com domain
[example.com]
# Map hosts to their motley_cue endpoints
login.example.com = https://login.example.com:8443
compute.example.com = https://login.example.com:8443

# Wildcard matching is supported
*.dev.example.com = https://dev-login.example.com:8443

# Override CA keys for this group (optional)
#user-ca-privkey = /etc/oinit-ca/example-user-ca
#user-ca-pubkey  = /etc/oinit-ca/example-user-ca.pub

# Another host group with different settings
[university.edu]
hpc.university.edu = https://auth.university.edu:8080
*.lab.university.edu = https://auth.university.edu:8080

Key Configuration Options#

  • host-ca-privkey/pubkey: SSH CA keys for signing host certificates (not currently used)
  • user-ca-privkey/pubkey: SSH CA keys for signing user certificates
  • cert-validity: Certificate lifetime
    • token: Inherit from OIDC token expiration
    • Number: Fixed duration in seconds
  • cache-duration: How long to cache motley_cue user info responses

Generating CA Keys#

# Generate user CA key pair
ssh-keygen -t ed25519 -f /etc/oinit-ca/user-ca -C "oinit User CA"

# Generate host CA key pair (for future use)
ssh-keygen -t ed25519 -f /etc/oinit-ca/host-ca -C "oinit Host CA"

# Set appropriate permissions
chmod 600 /etc/oinit-ca/*-ca
chmod 644 /etc/oinit-ca/*.pub

Running the Server#

Command Line#

systemctl start oinit-ca.service

Certificate Details#

Certificate Properties#

Generated certificates include:

  • Type: User certificate
  • Key ID: oinit@hostname (for identification in logs)
  • Principals: oinit and the username from motley_cue
  • Validity: Based on configuration (token expiry or fixed duration)
  • Critical Options: force-command set to oinit-switch <username>
  • Extensions: Standard SSH permissions (agent forwarding, port forwarding, etc.)

Security Features#

  • Certificates are tied to specific hosts via the Key ID
  • Force command ensures users can only run oinit-switch
  • Short validity periods limit exposure
  • Each certificate has a unique public key

Integration with motley_cue#

The CA server relies on motley_cue for:

  1. Token Validation: Verifying OIDC tokens are valid and not expired
  2. User Information: Getting the local username for the certificate
  3. Authorisation: Ensuring users are in “deployed” state
  4. Provider Discovery: Learning which OIDC providers are supported

Troubleshooting#

Common Issues#

  1. “motley_cue is not reachable”

    • Check the motley_cue URL in configuration
    • Verify network connectivity
    • Check motley_cue service status

  2. “User is not authorized or suspended”

    • Token may be expired
    • User account may be suspended in motley_cue
    • Token may not have required scopes

  3. “Unknown host”

    • Host is not configured in any host group
    • Check configuration file for typos
    • Verify wildcard patterns match correctly

  4. Certificate not accepted by SSH server

    • Ensure SSH server trusts the CA public key
    • Check certificate validity period
    • Verify principals match SSH configuration

Debugging#

Enable debug logging by setting Gin mode: 

export GIN_MODE=debug
oinit-ca 0.0.0.0:8443 /etc/oinit-ca/config.ini

Security Considerations#

  1. CA Key Protection: Keep CA private keys secure and limit access
  2. Network Security: Use HTTPS/TLS for the API endpoint in production
  3. Token Validation: The CA doesn’t validate tokens directly - it trusts motley_cue
  4. Host Verification: Ensure motley_cue endpoints in config are correct
  5. Certificate Scope: Certificates are only valid for the specific host requested

Server Side: SSH Server#

On the ssh-server, the tools oinit-switch and oinit-shell are required.

oinit-switch is a secure user switching utility that serves as the entry point for SSH connections authenticated with oinit certificates. It’s specified as a force-command in SSH certificates to ensure controlled access.

oinit-shell is a restricted shell that acts as a security layer between SSH and oinit-switch. It the shell for oinit user and ensures it can only execute the force-command specified in SSH certificates and prevents interactive shell access.

How It Works#

The oinit user needs a login shell for SSH connections to work, but should never provide interactive access. oinit-shell serves as this restricted shell by:

  1. Only accepting commands in the format: oinit-shell -c 'oinit-switch <username>'
  2. Rejecting any other command attempts
  3. Preventing interactive shell sessions

With this the login flow is running as:

  1. SSH certificates issued by oinit-ca contain: force-command="oinit-switch <username>"
  2. When users connect as the oinit user, oinit-shell runs oinit-switch instead of their requested command
  3. oinit-switch validates the request and uses su to switch to the target user
  4. The final user session runs with proper privileges and environment

Security Features#

  • Command Validation: Only allows oinit-switch with exactly one argument
  • No Interactive Access: Rejects any attempt at interactive shell
  • Minimal Attack Surface: Simple validation logic with no complex parsing
  • Process Replacement: Uses execve to avoid lingering processes
  • System User Protection: Refuses to switch to system users (UID below 100)
  • User Validation: Only allows switching from the oinit user or to the one specified in the ssh-cerficate
  • TTY Detection: Prevents command execution with TTY allocation to avoid security holes
  • Minimal Privileges: Uses execve to replace itself, minimizing resource usage

Installation#

Prerequisites#

  • oinit system user must exist
  • Target users must exist on the system
  • su command must be available

Packages#

apt install oinit-openssh

yum install oinit-openssh

zypper install oinit-openssh

Notice

oinit-shell is installed to run with setuid privileges.

Usage#

oinit-switch and oinit-shell are not meant to be run directly. They are invoked automatically via SSH force-command, which is encoded into ssh-certificates issues by oinit-ca.

Security Considerations#

  1. Force Command: Always specified in certificates to prevent bypass
  2. TTY Restrictions: Commands run without TTY to prevent injection attacks
  3. User Validation: Strict checks on source and target users
  4. No Direct Execution: Users cannot run oinit-switch directly

Last change: Aug 18, 2025 11:21:32