Giving SSH Access to a Remote Machine Securely

Giving SSH access to a remote machine is a very common need, especially when working in a team, handling production servers, or collaborating with other engineers in different locations. Although it sounds simple, wrong practices — like sharing passwords among team members via chat or shared documents — can create serious security risks that are hard to trace. This article discusses how to properly give SSH access to a remote machine using SSH key authentication, complete with configuration steps and best practices commonly used in the professional world.

What Is SSH and Why Use SSH Keys?

SSH (Secure Shell) is an encrypted network protocol that lets you access and control a remote machine through a terminal securely — all communication between client and server is encrypted, so it can’t be intercepted in plain text. When you want to give access to someone else, sharing a password isn’t a good solution for three reasons: the password can leak or be re-shared without your knowledge, it’s hard to trace who actually accessed the server if everyone uses the same credentials, and it’s hard to revoke one person’s access without changing the password for everyone else who also uses it.

The recommended solution is SSH key authentication, where each user has a unique identity consisting of a private key and public key pair. With this approach, revoking one person’s access is as simple as removing one line entry, without affecting anyone else’s access.

SSH Key Basics

SSH key authentication works on asymmetric cryptography, consisting of two parts:

• Private key — stored on the user’s computer, must never be shared with anyone
• Public key — stored on the server, safe to share and register

sequenceDiagram
    participant User as User (Client)
    participant Server as Remote Machine

    User->>Server: SSH connection request
    Server->>User: Encrypted challenge (using the public key)
    User->>User: Decrypt with the private key
    User->>Server: Send the decryption result
    Server->>Server: Verify the result
    Server-->>User: Access granted if it matches

When a user tries to log in, the server doesn’t ask for a password — the server sends an encrypted challenge using the registered public key. Only the owner of the matching private key can decrypt that challenge and prove their identity. Because the private key is never sent over the network, this process is far more secure than password-based authentication, which is vulnerable to sniffing or brute-force.

The public key and private key are always created as a pair, and can’t be used separately with a key from a different pair. The server only stores the public key — even if that server gets hacked, a leaked public key can’t be used to log in anywhere, because without the matching private key, the challenge can never be decrypted.

There are several algorithms commonly used to create SSH keys, and each has different characteristics:

This article uses RSA as an example because of its widest support across various systems, including older systems that may not yet support Ed25519. For more modern systems, ssh-keygen -t ed25519 can be a faster choice with equal or better security.

Steps to Give SSH Access

Ask the User for Their Public Key

The user who wants access must first create an SSH key on their own machine:

ssh-keygen -t rsa

This command produces two files:

• ~/.ssh/id_rsa — private key, stays on the user’s machine
• ~/.ssh/id_rsa.pub — public key, to be shared with the server

The public key can be displayed with:

cat ~/.ssh/id_rsa.pub

Ask the user for the contents of this public key — not the file, just the text, for example sent via internal chat or a ticketing system. The public key content is a single-line string starting with ssh-rsa (or ssh-ed25519 if using the Ed25519 algorithm), followed by base64 characters, and usually ending with a comment like the user’s email.

Log In to the Remote Server

As an administrator, enter the server using an account that already has access:

ssh user@remote-ip

Replace user and remote-ip according to the actual server configuration. At this stage you yourself still log in using the existing method — whether a password or your own SSH key — because the goal is to add access for a new user, not change your own login method.

Add the Public Key to the Server

After successfully entering the server, open the authorized_keys file in the .ssh directory of the account being granted access:

nano ~/.ssh/authorized_keys

If the .ssh directory doesn’t exist yet (for example for a newly created user), create it first:

mkdir -p ~/.ssh

Paste the user’s public key on a new line in that file — each line in authorized_keys represents one allowed key, so you can add several public keys for different users in the same file at once. As a faster alternative without opening an editor, use:

echo "PASTE_PUBLIC_KEY_HERE" >> ~/.ssh/authorized_keys

The >> operator appends a new line at the end of the file without deleting the existing contents — important to note, because using > (single sign) instead of >> will overwrite the entire file and remove the access of other users already registered before.

Make Sure File Permissions Are Correct

Wrong permissions are the most common cause of SSH login failures even when the public key is correctly registered. SSH deliberately rejects connections if directory or file permissions are too open, as a protection mechanism against other users on the same machine who could potentially modify the contents of authorized_keys. Use the following permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

chmod 700 makes the .ssh directory accessible (read, write, execute) only by its owner. chmod 600 makes the authorized_keys file readable and writable only by its owner, with no access for group or other users. Without these correct permissions, the SSH daemon (sshd) will reject the login request even though the key being used is actually valid.

Test the SSH Access

After all the steps above are complete, the user can try to log in:

ssh user@remote-ip

If the configuration is correct, the user will be able to log in without entering a password — the SSH client on the user’s machine will automatically try the private keys in ~/.ssh/, and once it finds a pair matching the public key on the server, authentication succeeds without further manual interaction.

If the login still asks for a password after the public key was added, recheck the ~/.ssh and authorized_keys permissions on the server, and make sure the server’s sshd_config allows PubkeyAuthentication yes. Error logs can usually be seen in more detail by running ssh -v user@remote-ip on the client side to see which stage failed.

Why This Approach Is More Secure

Compared to sharing passwords, SSH key authentication gives several concrete advantages:

Each user has a unique identity because everyone has their own private key that isn’t shared with anyone, so server activity can be more easily linked to a specific individual when combined with good logging. Access can also be revoked quickly just by removing one public key line from authorized_keys, without affecting other users. SSH keys are also far more resistant to brute-force attacks than passwords, because the space of possible key combinations is much larger and practically impossible to guess by trial and error. This approach is the industry standard for managing server access, whether for small teams or large-scale production infrastructure.

Limiting Access per Key

Besides simply allowing or denying login, the authorized_keys file also supports additional options that limit what a specific key can do. This is useful when you want to give restricted access — for example only to run one specific command, without opening full shell access to the server.

command="rsync --server -vlogDtprz --partial . /var/backup/",no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAA...rest_of_public_key... user@client

The command="..." option at the start of the line forces that key to only be able to run the specified command, regardless of what command the user actually sends at login — suitable for scenarios like automatic backups via rsync or scp, where the user (or service account) never needs interactive shell access. The no-port-forwarding and no-X11-forwarding options close tunneling features irrelevant for that scenario, while no-pty prevents this key from opening an interactive terminal session at all.

This approach is often used for service accounts — accounts used by automated systems (CI/CD pipelines, cron jobs, backup scripts) rather than humans. By limiting a service account key to only run one specific command, the impact if that key leaks becomes far smaller than a key with full shell access — an attacker who obtains that key remains locked to one specific predefined command, not getting free access to the entire file system and processes on the server.

Tips and Best Practices

• Never share a private key with anyone, including trusted teammates
• Use a passphrase on the private key as an extra security layer in case the private key file leaks
• Remove the public key from authorized_keys as soon as access is no longer needed — for example when someone leaves the team or a project ends
• Limit access by using a separate user for each person, don’t always use the root account for daily operations

A passphrase on the private key works as a second layer: even if the private key file is successfully stolen, the attacker still has to crack that passphrase before the key can be used. Meanwhile, avoiding root for daily operations helps limit the impact if one user account is compromised — a regular account with restricted privileges is far safer than an account with full access to the entire system.

As an additional checklist before considering the setup complete, make sure the following are also taken care of:

□ Password authentication disabled in sshd_config (PasswordAuthentication no)
□ Root login via SSH disabled (PermitRootLogin no)
□ Each user/service account has their own key, no key sharing
□ authorized_keys reviewed periodically, unused keys removed
□ Service accounts use the command= option to limit what can be run

Disabling PasswordAuthentication in sshd_config is actually an advanced step that completes the SSH key setup — after all users have switched to SSH keys, password login can truly be turned off at the server level, so brute-forcing passwords is no longer a relevant attack vector at all.

Summary

• SSH key authentication replaces passwords with a private key (on the client) and public key (on the server) pair that’s more secure and easier to audit.
• The public key is added to the ~/.ssh/authorized_keys file on the server — each line represents one allowed key.
• Correct permissions are mandatory — chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys, or SSH will reject the connection even with a valid key.
• Revoking one user’s access is as simple as removing one line in authorized_keys, without affecting other users.
• Never share a private key; use a passphrase as an additional security layer.
• Avoid using the root account for daily operations — use separate users with restricted privileges.
• This approach is the industry standard and applies to all environments — development, staging, and production.