SSH and Remote Access Configuration in Linux

Almost every server in the world is managed remotely. Nobody walks into a data center, plugs in a monitor, and types commands. Instead, administrators connect over the network using SSH (Secure Shell) — the standard, encrypted way to control one computer from another.

If you are learning Linux system administration, this is one of the first real skills you need. In this guide, we will cover what SSH is, how it works under the hood, how to install and configure the SSH server on Debian and Ubuntu, how to set up passwordless login with keys, and — most importantly — how to lock down your server so it is safe on the public internet.

What Is SSH and Why Does It Matter?

SSH is a protocol that gives you a secure, encrypted terminal on a remote machine. When you connect to a server with SSH, it is as if you are sitting in front of it with a keyboard — except you could be on the other side of the world.

Before SSH, people used tools like telnet, which sent everything — including your password — in plain text. Anyone watching the network traffic could read it. SSH was created to fix exactly this problem.

With SSH, everything is encrypted:

• Your password and commands
• The files you transfer
• The entire session traffic

SSH is used for far more than terminals, though. Once you know it, you will use it for:

• Administering remote servers (your daily driver)
• Securely copying files (scp, rsync, sftp)
• Tunneling other traffic through an encrypted channel (port forwarding)
• Git operations with remote repositories
• Automated deployments from CI/CD pipelines

How SSH Works: The Basic Flow

Before the commands, it helps to understand the mechanics. An SSH connection always has two sides:

• SSH client — the machine you are sitting at (your laptop, or even Termux on a phone)
• SSH server (sshd) — the remote machine you want to log into

The flow looks like this:

1. You run ssh user@server-ip on the client.
2. The client and server perform a key exchange and agree on encryption keys for this session.
3. The server proves its identity using its host key (stored in /etc/ssh/ on the server).
4. The client verifies your identity — with a password or, better, a public/private key pair.
5. An encrypted session opens, and you get a shell on the remote machine.

Note: The first time you connect to any server, SSH asks you to confirm the server's fingerprint. It then saves it to ~/.ssh/known_hosts. From then on, if that fingerprint ever changes, SSH refuses to connect and warns you loudly — that is protection against man-in-the-middle attacks, not a bug.

Step 1: Installing the SSH Server

On a fresh Debian or Ubuntu server, the SSH server is usually already installed. Check its status:

sudo systemctl status ssh 

If it is not installed:

sudo apt update sudo apt install openssh-server -y 

Then confirm it is running and enabled at boot:

sudo systemctl enable --now ssh sudo systemctl status ssh 

● ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled) Active: active (running) since Sat 2026-10-17 09:00:00 UTC 

The SSH server listens on port 22 by default. Verify it:

sudo ss -tlnp | grep sshd 

LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=812,fd=3)) 

Note on Ubuntu: Since Ubuntu 22.10, the service may be named ssh.socket instead of ssh.service (socket activation). The commands above work either way, but if you ever wonder why sshd is not listed as a running service, that is why.

Step 2: Connecting for the First Time

From your client machine:

ssh yeasin@203.0.113.10 

The authenticity of host '203.0.113.10' can't be established. ED25519 key fingerprint is SHA256\:xxxxxxxxxxxxxxxxxxxxxxxxxxx. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes yeasin@203.0.113.10's password: 

Enter the password, and you are in. If you can ping the server but the connection is refused, the most common cause is a firewall — which brings us to an important step:

sudo ufw allow ssh 

On Debian, ufw may not be installed; install it with sudo apt install ufw. Without this rule, remote SSH traffic never reaches the server.

Step 3: Key-Based Authentication (Do This First)

Passwords can be guessed, reused, or leaked. SSH keys solve this. A key pair consists of:

• A private key (~/.ssh/id_ed25519) — never leaves your machine. Never share it, never copy it to a server.
• A public key (~/.ssh/id_ed25519.pub) — safe to share; you place it on every server you want to access.

The server can verify you own the private key without you ever sending it. That is the whole trick: the private key proves your identity without ever traveling over the network.

Generate a key pair

On your client machine:

ssh-keygen -t ed25519 -C "yeasin@laptop" 

Generating public/private ed25519 key pair. Enter file in which to save the key (/home/yeasin/.ssh/id_ed25519): Enter passphrase (empty for no passphrase): Your identification has been saved in /home/yeasin/.ssh/id_ed25519 Your public key has been saved in /home/yeasin/.ssh/id_ed25519.pub 

• -t ed25519 — the modern, fast, and secure key type. Prefer it over RSA.
• The passphrase protects your private key if your laptop is stolen. Recommended.

Copy the public key to the server

ssh-copy-id yeasin@203.0.113.10 

That is it. Log in again and you will not be asked for a password:

ssh yeasin@203.0.113.10 

Tip: If ssh-copy-id is unavailable (for example on Termux), do it manually by appending your public key to the server's ~/.ssh/authorized_keys file. Make sure the .ssh directory has permission 700 and the file has 600 — SSH ignores keys if the permissions are too open.

Step 4: Understanding and Configuring sshd

All server-side configuration lives in one file:

/etc/ssh/sshd_config 

Before changing anything, create a backup:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak 

After every change, test your configuration before restarting:

sudo sshd -t 

No output means everything is fine. Then apply the changes:

sudo systemctl restart ssh 

Warning: Never close your current SSH session while testing new SSH settings. Keep it open, open a second terminal, and confirm you can still log in. If you locked yourself out, your open session is your rescue line.

The most important settings

sudo nano /etc/ssh/sshd_config 

Port 2222 PermitRootLogin no PubkeyAuthentication yes PasswordAuthentication no AllowUsers yeasin MaxAuthTries 3 

Each of these lines removes one attack path:

• PermitRootLogin no — attackers always try root first. Even with a good password, you do not want them hammering the most powerful account.
• PasswordAuthentication no — once your keys work, this is the single biggest security upgrade. Nothing can be brute-forced if passwords are not accepted.
• AllowUsers — even a valid password for an unused account (like guest) is now useless.

Note: On modern Ubuntu (22.04+), password authentication may be controlled by a drop-in file at /etc/ssh/sshd_config.d/*.conf, which overrides the main config. Check with grep -ri PasswordAuthentication /etc/ssh/ to make sure your change actually takes effect.

Changing the default port

Changing the port (e.g., Port 2222) is security through obscurity — it reduces automated bot noise dramatically, but a targeted attacker will scan all ports anyway. Treat it as noise reduction, not real security:

sudo ufw allow 2222/tcp 

ssh -p 2222 yeasin@203.0.113.10 

Order matters: open the firewall for the new port before restarting sshd, or you will lock yourself out.

Step 5: The Client-Side Config File

Tired of typing ssh -p 2222 yeasin@203.0.113.10? SSH has a built-in solution. Create ~/.ssh/config on your client:

nano ~/.ssh/config 

Host myserver HostName 203.0.113.10 User yeasin Port 2222 IdentityFile ~/.ssh/id_ed25519 

Now connecting is simply:

ssh myserver 

This also makes scp, rsync, and sftp use the same settings automatically. If you manage more than one server, this file will save you hours.

Step 6: Extra Hardening for Public Servers

If your server has a public IP, bots are already knocking on port 22 — check for yourself:

sudo journalctl -u ssh | grep -i failed | tail 

You will likely see login attempts for root, admin, and test from IP addresses all over the world. Two more layers stop this completely.

1. Fail2ban — automatic attacker blocking

Fail2ban watches the logs and bans IPs after repeated failed attempts:

sudo apt install fail2ban -y sudo systemctl enable --now fail2ban sudo fail2ban-client status sshd 

Status |- Number of matches: 14 |- Currently banned: 2 `- Total banned: 9 

On Debian and Ubuntu, fail2ban ships with a ready-made sshd jail; default is 5 failed attempts leading to a 10-minute ban. Adjust it in /etc/fail2ban/jail.local.

2. Firewall — allow only what you need

sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw allow 2222/tcp sudo ufw enable sudo ufw status verbose 

Warning: Before running ufw enable on a remote server, make sure your SSH port is allowed. Locking yourself out of a remote machine is a rite of passage — but not one you need to experience.

Common Mistakes and Troubleshooting

1. "Connection refused"
The server is reachable, but nothing is listening on that port. Check: is ssh running (sudo systemctl status ssh)? Is the firewall allowing the port? Are you connecting to the right port?

2. "Connection timed out"
Packets are not reaching the server. Check the IP address, cloud provider security groups (AWS/GCP/DigitalOcean firewalls are separate from ufw), and any network firewall in between.

3. Key login silently falls back to password
Usually permissions. On the server:

chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys 

And check the logs — they tell you exactly why a login failed:

sudo journalctl -u ssh -n 20 

4. Locked yourself out after changing the port
This is why you never close the working session during changes. If it does happen: on a VPS, use the provider's web console to fix sshd_config; on a local machine, plug in a keyboard and monitor.

5. "Too many authentication failures"
Your client may be offering many keys. Fix it in ~/.ssh/config with IdentitiesOnly yes on that host, so only the right key is offered.

Summary

• SSH is the encrypted standard for remote administration — everything you type and transfer is protected end to end.
• Install the server with openssh-server, and immediately allow it through ufw.
• Replace password login with ed25519 key pairs as soon as possible — it is both more secure and more convenient.
• Harden the server in /etc/ssh/sshd_config: PermitRootLogin no, PasswordAuthentication no, and AllowUsers for your accounts.
• Always test with sshd -t and keep your current session open while applying changes.
• Use ~/.ssh/config on the client to turn long commands into ssh myserver.
• Add fail2ban on public servers; bots are already trying your door.

Remote access is the gateway skill: once SSH feels natural, managing servers, deploying applications, and automating infrastructure all start from that same ssh command.