How to Install and Configure SSH with Public and Private Keys on Linux

Secure Shell (SSH) is a vital tool for system administrators and developers needing encrypted access to remote machines. Rather than relying on passwords, you can use SSH keys to authenticate users securely.

In this step-by-step guide, you'll learn how to install the SSH service and configure public/private key authentication on Debian-based systems such as Ubuntu.

System Requirements

Before starting, ensure you're using a Debian-based Linux distribution such as Ubuntu or Debian. This guide assumes you have administrative privileges (sudo access) on the machine.

Update and Install the SSH Server

Start by updating your system's package index to ensure you have access to the latest packages:

sudo apt-get update

Then install the OpenSSH server package, which enables SSH access to your machine:

sudo apt-get install openssh-server

After installation, verify that the SSH service is running:

sudo systemctl status ssh

If your system doesn’t support systemctl, use this alternative:

sudo service ssh status

Connect to the SSH Server

Once SSH is installed and running, you can connect to the server using:

ssh username@ip_address

Replace username with your Linux user and ip_address with your server's IP. You can find the IP address by running:

ip addr

Look for the IP address listed after the keyword inet.

Configure SSH with Public and Private Keys

To enhance security and eliminate the need for passwords, you can configure SSH to use a key pair (public and private keys) for authentication.

Generate SSH Keys

On your local machine, generate a 4096-bit RSA key pair with the following command:

ssh-keygen -t rsa -b 4096 -C "your_key_comment"

When prompted, accept the default file location:

Enter file in which to save the key (/home/your-user/.ssh/id_rsa):

Next, enter a secure passphrase. This is optional during development but highly recommended for production environments.

Copy the Public Key to the Remote Server

To authorize your local machine, copy the public key to the remote server:

ssh-copy-id username@ip_address

You’ll be prompted for the remote user’s password. If successful, you’ll see:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'username@server_ip_address'"
and check to make sure that only the key(s) you wanted were added.

Alternative Key Transfer Method

If ssh-copy-id is not available, use this manual method:

cat ~/.ssh/id_rsa.pub | ssh username@ip_address "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

This command securely appends your public key to the authorized_keys file on the server.

Secure the SSH Server Configuration

To enforce key-based login and disable password authentication, edit the SSH configuration file on the remote server:

sudo nano /etc/ssh/sshd_config

Update or add the following lines:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

These settings disable all forms of password login, requiring users to authenticate using SSH keys only.

From now on, only users with the correct private key (and optional passphrase) can access the server.

Using SSH Keys Across Multiple Devices

If you use multiple devices, you’ll need to repeat the key generation and transfer steps for each device. To avoid managing multiple keys, consider using an SSH agent or combining your public keys in the authorized_keys file.

Conclusion

Setting up SSH with public and private keys is a fundamental step for securing remote Linux servers. By disabling password-based authentication, you greatly reduce the risk of brute-force attacks.

This guide has walked you through the installation of the SSH service, generation of RSA keys, and configuration of a secure connection—ensuring your server remains protected and accessible only to authorized users.

For further learning, explore the official OpenSSH documentation or consult Ubuntu’s SSH server guide.