Hardening SSH for Internet-Facing Servers

The moment a Linux server gets a public IP, random scanners start hammering port 22. Within hours you will see thousands of login attempts against root and common usernames. SSH is your only admin doorway, so its hardening is the single highest-leverage security task on any internet-facing box. This guide walks through the three layers that actually matter: strong key authentication, a deliberately minimal sshd configuration, and fail2ban to keep persistent attackers from brute-forcing your way through. Each step is concrete and safe to run on an Ubuntu 24.04 or Debian-style VPS.

Generate a Strong Key Pair

Passwords die with the first script that guesses them. A private SSH key gives an attacker essentially nothing to work with unless they compromise your workstation. Generate an ed25519 key on your local machine — ed25519 is small, fast, and considered secure by default in modern OpenSSH versions — then copy the public half to the server.

ssh-keygen -t ed25519 -a 64 -C "you@laptop" -f ~/.ssh/id_ed25519
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@YOUR_SERVER_IP

The -a flag raises the KDF iteration count, which makes it dramatically slower for someone to brute-force the passphrase on your private key. Never skip the passphrase: a compromised laptop with an unencrypted key is a compromised server. Keep the private key only on machines you control and never commit it to a repository.

Create a Real Admin User

Logging in as root is convenient and wrong in equal measure. A normal user with sudo gives you an audit trail for every privileged action and lets you revoke access without touching the root account. Create the user, grant sudo, and test the key before you ever disable password auth.

sudo adduser deploy
sudo usermod -aG sudo deploy
sudo mkdir -p /home/deploy/.ssh
sudo cp ~/.ssh/authorized_keys /home/deploy/.ssh/
sudo chown -R deploy:deploy /home/deploy/.ssh
sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys

The file permissions on .ssh matter. A world-readable authorized_keys or an .ssh directory owned by root will make the SSH daemon refuse to use your key, forcing you to rely on passwords you are about to turn off. Get the ownership and mode bits right now and you will avoid a lockout round-trip.

Harden sshd with a Drop-in

OpenSSH on Ubuntu reads every file in /etc/ssh/sshd_config.d before it falls back to the main config. Put your hardening in a numbered drop-in so it is obvious, reversible, and never mangled by a config rewrite. Always run sshd -t before restarting the service — a bad directive is the classic way to shut the door on yourself.

sudo tee /etc/ssh/sshd_config.d/50-hardening.conf > /dev/null <<'EOF'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
LoginGraceTime 30
AllowUsers deploy
EOF
sudo sshd -t
sudo systemctl restart ssh

PermitRootLogin no closes the most-attacked account on the box, and PasswordAuthentication no removes the entire class of dictionary attacks. AllowUsers deploy is a whitelist: everyone else is refused at the daemon level, before fail2ban even sees the attempt. Open a second terminal and confirm deploy can still log in with the key before you end the current session.

Block the Noise with fail2ban

Even with password auth off, attackers keep probing, and every connection wastes CPU and log space. fail2ban watches the authentication log, counts failed attempts per source IP, and inserts an iptables rule that drops the offender for a configurable window. On Ubuntu it ships with an sshd jail ready to go.

sudo apt install -y fail2ban
sudo systemctl enable --now fail2ban
sudo fail2ban-client set sshd bantime 1h
sudo fail2ban-client status sshd

The bantime command duck-tapes a sane default without touching config files. For a serious internet-facing server, raise the bantime in /etc/fail2ban/jail.local so well-known blacklist sources stay banned far longer than attackers are willing to keep trying. Combine it with the whitelist and you filter most junk before your CPU ever negotiates a key exchange.

Verify the Whole Chain

Hardening you cannot see is hardening you cannot trust. Check that the daemon actually loaded your directives and that fail2ban sees the sshd jail as active.

sudo sshd -T | grep -E "permitrootlogin|passwordauthentication|allowusers"
sudo fail2ban-client status

If every login attempt from the outside is refused before it authenticates, and the only path in is your key, you have a defensible boundary. Keep the admin session you are already in open until you are certain, then log out and back in a couple of times to be sure nothing regressed.

Takeaway

Three layers cover the practical attack surface: key-only authentication, a minimal sshd that refuses root and unused auth methods, and fail2ban mopping up the endless scanner noise. Each layer is cheap and individually verifiable, and together they turn SSH from the most attacked service on your box into one you barely think about. You can spin up an Ubuntu 24.04 instance on Netbay in under 60 seconds and follow along from a fresh host — netbayhosts.in.