How to connect to a Linux VPS over SSH for the first time

The first thing you do on a fresh Linux VPS is open a terminal and connect to it over SSH. It sounds trivial, but the very first connection is where most people hit their first wall: a permission error, a wrong username, or a port that is silently closed. This guide walks you through how to connect to a Linux VPS over SSH for the first time, assuming the server runs Ubuntu 24.04 or Debian 12 and you have the root password from your provider.

Key takeaways

• Generate an ED25519 key pair locally, never reuse an RSA key from years ago.
• Copy the public key with ssh-copy-id, then log in with the key before you disable password auth.
• The dreaded "Permissions 0644" error means your private key must be readable only by you, chmod 600 fixes it.
• Change the SSH port and disable root login only after you have verified key-based login works.

Prerequisites

• A Linux VPS with Ubuntu 24.04 or Debian 12, reachable on the internet.
• Root password or a sudo user provided by the hosting panel.
• A local machine with OpenSSH installed: Linux, macOS, or Windows 10/11 with the OpenSSH client enabled.
• The public IP address of the server, visible in your provider control panel.

If you do not have a server yet, rent a Linux VPS with a dedicated IPv4 and full root access, then come back here. Everything below assumes you hold the root password and can reach the IP from your network.

Why the first SSH connection deserves care

SSH is the door to your server. If the door is left open with a weak password, bots will find it within hours. Automated scanners sweep the whole IPv4 space continuously and try common username-password pairs against port 22. Your fresh VPS with a Vietnam VPS with dedicated IPv4 is no exception. That is why the first connection is not just about getting in, it is about setting up the habit of key-based authentication from the start.

Password logins are convenient but fragile. A strong key pair is practically unguessable. Once you copy your public key to the server and disable password authentication, the scanning bots have nothing to try. The few extra minutes on the first day save you from a compromised box later. I do this on every server I touch, and I recommend you do the same.

第一次连接 VPS 时立即配置密钥登录，可避免密码暴力破解。

Configuring key-based login on your very first VPS connection prevents password brute-force attacks.

Step 1 - Generate an SSH key pair on your local machine

Open a terminal on your local computer. If you are on Windows, open PowerShell. Check whether you already have a key pair; if you do, you can skip generation. Run this on your local machine, not on the server:

ls -la ~/.ssh/id_ed25519*

If the files do not exist, generate a new ED25519 key pair. ED25519 is faster and more secure than the older RSA 2048, and it produces a much shorter key. Use your email as a comment so you recognize the key later:

ssh-keygen -t ed25519 -C "[email protected]"

You will be asked where to save the key. Press Enter to accept the default location ~/.ssh/id_ed25519. Then set a passphrase. A passphrase encrypts the private key on disk, so even if someone steals the file, they cannot use it without the passphrase. I recommend setting one; the small cost of typing it each session is worth the protection. If you use an SSH agent, you only type it once per boot.

Verify the two files were created:

ls -la ~/.ssh/id_ed25519*

You should see two files: id_ed25519 (private) and id_ed25519.pub (public). The public key is what you will copy to the server. The private key stays on your machine, always.

Step 2 - Find the right username and IP address

Before connecting, you need two things: the username and the IP. On most images from providers, the default user is root. Some images create a user like ubuntu or debian instead. Check the welcome email or the control panel of your VPS provider to see which user was created.

Common default usernames by OS:

The IP address is shown in your provider panel as "IP Address" or "Public IP". It looks like 203.113.25.10. Copy it, you will use it in the next step.

Step 3 - Connect over SSH with a password for the first time

Now make the first connection. Replace your_user with your actual username and 203.113.25.10 with your server IP:

ssh [email protected]

The first time you connect, SSH asks you to confirm the host key fingerprint. This is normal. Type yes and press Enter. Then enter the password from your provider. You are now inside the server. Run a quick check that everything is alive:

cat /etc/os-release | head -2

You should see the OS name and version. You are connected.

A quick note before you go further: I strongly recommend changing the root password right now if you are logging in as root with the provider password. Run passwd and set something long and unique. The password from the panel is often shared or guessable, and you do not want it lingering.

Step 4 - Copy your public key to the server

Exit the server session with exit or Ctrl-D. Back on your local machine, copy the public key using ssh-copy-id. This tool appends your public key to ~/.ssh/authorized_keys on the server and sets the right permissions automatically:

ssh-copy-id [email protected]

You will be prompted for the server password one last time. After it succeeds, test key-based login:

ssh [email protected]

This time you should get in without a password prompt (or only the passphrase prompt for your local key). If it works, your public key is in place. If you do not have ssh-copy-id on your system, append the key manually:

cat ~/.ssh/id_ed25519.pub | ssh [email protected] "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

This one-liner creates the .ssh directory, sets the right permissions on it, appends your key, and locks down the authorized_keys file. Permission mistakes here are the most common cause of login failures, so pay attention to the chmod values.

Step 5 - Understand and fix the classic permission errors

The most frequent error on a first connection is Permissions 0644 for '/home/you/.ssh/id_ed25519' are too open. SSH refuses to use a private key that other users on your machine can read. The fix is to tighten permissions on the private key file:

chmod 600 ~/.ssh/id_ed25519

On the server side, the authorized_keys file must not be writable by group or others. If you see Authentication refused: bad ownership or modes, check both the file and the home directory:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R your_user:your_user ~/.ssh

Replace your_user with your actual username. These three commands resolve the vast majority of SSH key login failures. I have fixed more broken servers with these three lines than with anything else.

Step 6 - Harden sshd before you close the password door

Now that key login works, disable password authentication. Edit the SSH daemon configuration:

sudo nano /etc/ssh/sshd_config

Set these three values:

PermitRootLogin prohibit-password
PasswordAuthentication no
KbdInteractiveAuthentication no

PermitRootLogin prohibit-password allows root to log in only with a key, never with a password. PasswordAuthentication no disables password logins for all users. Validate the configuration before restarting, a wrong value here locks you out:

sudo sshd -t

If the command outputs nothing, the config is valid. Restart the service:

sudo systemctl restart ssh

Then verify the service is running:

systemctl status ssh

You should see active (running). Open a second terminal and try to log in with your key one more time. If it works, you are safe. If you cannot log in, check the error and fix it before you close the current session.

For an even stronger setup, consider moving SSH to a non-standard port and adding fail2ban or CrowdSec to block brute-force attempts at the firewall level. A non-standard port stops most automated scans instantly.

Troubleshooting common first-connection failures

Connection refused. The SSH service is not running or the firewall drops the port. Check the service and the listening socket:

sudo systemctl status ssh
ss -tlnp | grep :22

If ss shows nothing, enable and start SSH: sudo systemctl enable --now ssh. On a fresh server the firewall may also block port 22. Open it with sudo ufw allow 22/tcp if you use ufw, or sudo firewall-cmd --add-service=ssh --permanent on AlmaLinux and Rocky.

Permission denied (publickey). Your public key is not in authorized_keys, or the file permissions are wrong. Re-run step 4 and verify the chmod values from step 5.

Connection timed out. The IP is wrong, the server is offline, or a network path blocks port 22. Confirm the IP in the panel and try to ping the server. Some hosting networks block ICMP, so a failed ping does not mean the server is down.

Host key verification failed. The server was reinstalled or you copied the wrong IP. Remove the old host key entry with ssh-keygen -R 203.113.25.10 and connect again.

FAQ

Related articles

• How to connect to a Linux VPS for the first time via SSH
• Hardening SSH on a new VPS: keys, ports, fail2ban, and CrowdSec
• User and sudo permissions management on a Linux VPS
• Set up nftables firewall on a VPS to replace iptables