Configuring Access to the Server by SSH Key

In the modern digital infrastructure, secure and automated server access is essential for system administrators, developers, and DevOps engineers. SSH key-based authentication is the most common and reliable way to do this. SSH keys improve security by eliminating password-based logins and enable streamlined automation, particularly when managing multiple servers or integrating CI/CD pipelines. This article provides a comprehensive exploration of configuring SSH key access to a server, offering insights from both practical and administrative viewpoints.

Understanding SSH and Key-Based Authentication

SSH, or Secure Shell, is a protocol that allows secure remote login and command execution on a remote server. SSH uses password authentication by default, but this method is susceptible to brute-force attacks and credential leaks. SSH key-based authentication is the clear choice: more secure and more convenient.

SSH keys come in pairs: a private key, which remains on the client machine and must be protected, and a public key, which is placed on the server in a special file. When a client attempts to connect to the server, the server checks if the corresponding public key exists and verifies the client using the private key. If everything matches, access is granted.

Managing public keys is simpler and safer than handling multiple user passwords, especially when automating access control across distributed infrastructure.

Generating SSH Keys

Before configuring a server for SSH key access, a key pair must be generated. This is usually done on the client machine using the ssh-keygen tool, which is available on all major Unix-like systems by default.

    ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

This command generates a 4096-bit RSA key pair with an optional comment for identification. The process prompts for a file location and a passphrase:

    Location: By default, the key is stored in ~/.ssh/id_rsa (private) and ~/.ssh/id_rsa.pub (public).
    Passphrase: Optional, but highly recommended for added security. It encrypts the private key locally.

Besides RSA, other algorithms like ED25519 or ECDSA may be used for better performance or stronger cryptographic properties. ED25519, for instance, offers shorter keys with robust security, and is increasingly favored for modern deployments.

Copying the Public Key to the Server

Once the key pair is generated, the public key must be copied to the target server. This can be done manually or with automation tools.

The simplest way to do it is with ssh-copy-id:

    ssh-copy-id -i ~/.ssh/id_rsa.pub user@remote_host

This command appends the public key to the ~/.ssh/authorized_keys file of the specified user on the server. It also ensures the correct file permissions are applied to the .ssh directory and the authorized_keys file—both of which are critical for successful authentication.

Alternatively, one can copy the public key manually:

    cat ~/.ssh/id_rsa.pub | ssh user@remote_host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

From a security perspective, it is essential that the ~/.ssh directory has 700 permissions and authorized_keys is set to 600. Otherwise, OpenSSH may refuse to use the key for authentication.

Disabling Password Authentication (Optional but Recommended)

Once SSH key access is working, it is considered good practice to disable password authentication entirely, thus minimizing the attack surface of the server. This is configured in the SSH daemon’s configuration file:

    sudo nano /etc/ssh/sshd_config

Locate or add the following lines:

    PasswordAuthentication no
    ChallengeResponseAuthentication no
    UsePAM no

After making the changes, restart the SSH service:

    sudo systemctl restart sshd

It is strongly advised to test the key-based access in a separate terminal before restarting the SSH service, especially when working on remote servers, to avoid locking yourself out due to misconfiguration.

Other useful articles in our Blog:

    - Transferring files using SSH in Linux

    - A quick guide to basic SSH setup on your Debian server: Mastering sshd_config

    - Linux scp Command and Examples of Its Use

    - How to Use Rsync to Sync Local and Remote Directories

Managing Multiple Keys and Users

In environments where multiple users need access to the same server, or where automation systems (such as Ansible or CI/CD runners) require distinct access, it becomes important to manage public keys carefully.

Each public key should be uniquely identifiable, and may be appended to the authorized_keys file along with a comment:

    ssh-rsa AAAAB3Nza... user1@laptop
    ssh-ed25519 AAAAC3Nza... deploy-bot@ci

For easier management, configuration management tools like Ansible, Puppet, or Chef can distribute and manage SSH keys automatically across multiple nodes. Additionally, tools like HashiCorp Vault or AWS Secrets Manager can securely store and rotate keys.

In shared environments, it’s wise to maintain a policy for SSH key expiration and regular audits. Keys that are no longer needed should be removed from authorized_keys to avoid stale access.

Using SSH Config for Easier Access

When accessing multiple servers frequently, the ~/.ssh/config file on the client machine helps simplify and manage SSH options. A typical entry might look like:

    Host web-server
    HostName 192.168.1.100
    User ubuntu
    IdentityFile ~/.ssh/id_rsa

This allows connecting using ssh web-server instead of specifying full parameters every time. Additionally, you can specify port numbers, jump hosts, and other useful options.

For example, to connect via a bastion host (jump host):

    Host internal-server
    HostName 10.0.0.2
    User dev
    IdentityFile ~/.ssh/dev_key
    ProxyJump bastion-host

Security Best Practices

Despite the inherent security of SSH keys, several best practices should be observed:

    - Use a strong passphrase for private keys, especially when stored on laptops or shared systems.
    - Avoid root login: Permit only regular users to authenticate and then escalate privileges with sudo.
    - Monitor SSH access using log files (/var/log/auth.log) or security tools like Fail2Ban, which block IPs after repeated failed attempts.
    - Limit IP ranges from which SSH access is allowed, using firewall rules or sshd_config with AllowUsers or Match Address.
    - Consider using hardware security modules (HSMs) or YubiKeys for storing private keys, especially in high-security environments.

In cloud environments, providers like AWS, Google Cloud, and Azure offer their own mechanisms for key injection and lifecycle management, which integrate with instance metadata or identity-based policies. Even in self-managed hosting, some platforms allow you to inject SSH keys at VM creation, which is ideal for ephemeral infrastructure.

Conclusion

Configuring server access using SSH key authentication is a foundational skill in systems administration and cloud computing. It enhances security, enables automation, and simplifies management across environments. Understand how to generate, install, and manage SSH keys to build more robust and secure infrastructures.

When paired with other security practices like two-factor authentication, restricted IP access, and automated provisioning, SSH key access becomes a core component of secure system design. Managing a personal VPS or a fleet of enterprise servers requires mastery of SSH key configuration for operational excellence in today's computing landscape.