Step-by-Step Linux Server Hardening Guide: Initial Security Setup

Deploying a fresh cloud VPS or dedicated bare-metal server is a matter of clicks. However, leaving a newly provisioned Linux instance in its default state exposes it to automated botnets and malicious actors within minutes of public exposure. Whether you run high-performance databases, microservices, or custom APIs, establishing a robust security posture from second one is non-negotiable.

This comprehensive Linux server hardening guide provides a production-grade, step-by-step blueprint to secure a fresh Linux server. We cover the three most popular enterprise and lightweight distributions: Debian/Ubuntu, AlmaLinux (RHEL-based), and Alpine Linux.

The Initial Linux Server Security Checklist

Step 1: Update the System and Create a Non-Root User

Never run your day-to-day services or SSH sessions directly as the root user. The first step to secure a fresh Linux server is updating the package index and creating a dedicated user with administrative privileges.

1.1 Update Existing Packages

Before configuring anything, pull down the latest security updates.

Debian/Ubuntu:

sudo apt update && sudo apt upgrade -y

AlmaLinux:

sudo dnf upgrade --refresh -y

Alpine Linux:

apk update && apk upgrade

1.2 Create a New Administrative User

Replace sysadmin with your preferred username.

Debian/Ubuntu:

# Create user and prompt for password
adduser sysadmin
# Add to sudo group
usermod -aG sudo sysadmin

AlmaLinux:

# Create user
useradd sysadmin
# Set password
passwd sysadmin
# Add to wheel group (RHEL equivalent of sudo)
usermod -aG wheel sysadmin

Alpine Linux: Alpine is designed to be ultra-minimal. It often uses doas instead of sudo to keep the footprint small.

# Install doas
apk add doas
# Create user
adduser sysadmin
# Add user to wheel group
addgroup sysadmin wheel
# Configure doas to permit wheel group to execute commands as root
echo "permit persist :wheel" > /etc/doas.d/doas.conf

Step 2: Secure SSH Access with Key-Based Authentication

Password-based authentication is highly vulnerable to brute-force attacks. Transitioning to SSH key-based authentication is the single most impactful hardening step you can take.

2.1 Generate and Copy SSH Keys

On your local machine (your laptop or workstation), generate a modern, high-security Ed25519 key pair:

ssh-keygen -t ed25519 -C "[email protected]"

Next, copy the public key to your new remote server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub sysadmin@<YOUR_SERVER_IP>

(For Alpine Linux, if ssh-copy-id is unavailable, manually append your public key to /home/sysadmin/.ssh/authorized_keys and run chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys).

2.2 Harden the SSH Daemon Configuration

Log into your server as the new sysadmin user and edit the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Modify or add the following directives to enforce strict key-based access:

# Disable root logins over SSH
PermitRootLogin no

# Disable password-based authentication
PasswordAuthentication no

# Permit only key-based authentication
PubkeyAuthentication yes

# Prevent empty passwords
PermitEmptyPasswords no

# Limit maximum authentication attempts
MaxAuthTries 3

# (Optional) Change default port to mitigate automated port scans
Port 2222

2.3 Restart the SSH Service

Apply the configuration changes. Do not close your current terminal window until you have successfully tested the connection in a new window!

Debian/Ubuntu:

sudo systemctl restart ssh

AlmaLinux:

sudo systemctl restart sshd

Alpine Linux:

sudo rc-service sshd restart

Step 3: Configure Host-Based Firewalls (UFW vs. FirewallD vs. nftables)

A robust firewall ensures that only explicitly permitted ports are accessible from the outside world.

Option A: Debian/Ubuntu (UFW Setup)

The Uncomplicated Firewall (UFW) is the standard, easy-to-use frontend for Debian-based systems.

# Install UFW if not present
sudo apt install ufw -y

# Set default rules
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow your custom SSH port (or port 22 if you didn't change it)
sudo ufw allow 2222/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Enable the firewall
sudo ufw enable

Option B: AlmaLinux (FirewallD Setup)

RHEL-based systems utilize FirewallD, a zone-based firewall daemon.

# Ensure firewalld is running and enabled
sudo systemctl enable --now firewalld

# Create a rule for your custom SSH port (assuming 2222)
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https

# Remove default SSH service if port was changed
sudo firewall-cmd --permanent --remove-service=ssh

# Reload firewall configuration
sudo firewall-cmd --reload

Option C: Alpine Linux (nftables / awall Setup)

Alpine Linux leverages modern nftables combined with awall (Alpine Wall) for policy-based configurations.

# Install awall and nftables
sudo apk add awall nftables

# Enable nftables on boot
sudo rc-update add nftables default

Create /etc/awall/private/custom.json:

{
  "description": "Base firewall rules",
  "zone": { "internet": { "iface": "eth0" } },
  "policy": [
    { "in": "internet", "action": "drop" },
    { "out": "internet", "action": "accept" }
  ],
  "filter": [
    { "in": "internet", "out": "_fw", "service": [ { "proto": "tcp", "port": 2222 }, "http", "https" ], "action": "accept" }
  ]
}

Activate the rules:

sudo awall enable custom
sudo awall activate

Step 4: Prevent Brute-Force Attacks with Fail2ban

Fail2ban monitors system logs and dynamically bans IP addresses exhibiting malicious behaviors, such as repeated failed login attempts.

4.1 Installation

Debian/Ubuntu:

sudo apt install fail2ban -y

AlmaLinux:

sudo dnf install epel-release -y
sudo dnf install fail2ban -y
sudo systemctl enable --now fail2ban

Alpine Linux:

sudo apk add fail2ban
sudo rc-update add fail2ban default

4.2 Configuration

Create a local jail configuration file (/etc/fail2ban/jail.local) to override defaults:

[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5

[sshd]
enabled  = true
port     = 2222
logpath  = %(sshd_log)s
backend  = %(sshd_backend)s

Restart the service to apply changes:

# Debian/Ubuntu/AlmaLinux
sudo systemctl restart fail2ban

# Alpine
sudo rc-service fail2ban restart

Step 5: Enable Automated Security Patches

In a production environment, keeping packages updated manually is an operational bottleneck. Automating security patches ensures that critical vulnerabilities are mitigated instantly.

5.1 Debian/Ubuntu (Unattended Upgrades)

sudo apt install unattended-upgrades apt-listchanges -y

# Reconfigure package to enable automated updates
sudo dpkg-reconfigure -plow unattended-upgrades

5.2 AlmaLinux (dnf-automatic)

sudo dnf install dnf-automatic -y

Edit /etc/dnf/automatic.conf and set:

upgrade_type = security
apply_updates = yes

Enable the systemd timer:

sudo systemctl enable --now dnf-automatic.timer

5.3 Alpine Linux (Cron-based Updates)

Because Alpine is built for containerized and low-overhead environments, it does not use a heavy background daemon. Instead, use a daily cron job. Create /etc/periodic/daily/apk-upgrade:

#!/bin/sh
apk update && apk upgrade --no-cache

Make the script executable:

sudo chmod +x /etc/periodic/daily/apk-upgrade

Summary & Next Steps

Securing your cloud infrastructure should never be an afterthought. By executing this initial Linux server security checklist, you eliminate over 99% of common automated threats.

Key Takeaways:

1. Never use root directly: Always route commands through a non-root user with sudo or doas.
2. Disable password auth: Enforce SSH key authentication using modern Ed25519 keys.
3. Minimize attack surface: Close all ports except those strictly required (e.g., SSH, HTTP, HTTPS) using UFW, FirewallD, or nftables.
4. Automate defense: Set up Fail2ban to block persistent scanners and enable automatic security updates to stay ahead of zero-day exploits.

For high-performance, secure cloud hosting, pair this guide with AZBrand's NVMe Cloud VPS. Built on enterprise-grade hardware with integrated DDoS protection and advanced virtualization, AZBrand provides the perfect, high-speed canvas for your hardened production workloads.