6 Server Security Practices That Aren't Obvious

Server hardening guides are everywhere and most of them are terrible. They list 150 controls without prioritizing, they copy each other, and they assume a universe where someone has infinite time to implement everything. Here are six practices that actually matter more than the other 144, that most teams skip, and that we implement on every server we build.

1. SSH: Disable Password Auth, Set AllowUsers, Restrict by Source

SSH is the first thing attackers probe. Leaving it in default configuration on a public IP is like putting a key under the doormat.

The changes that matter:

# /etc/ssh/sshd_config
PasswordAuthentication no
PermitRootLogin no
AllowUsers deploy admin
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2

• Disable password auth. Keys only. Passwords get brute-forced; keys don't.
• Disable root login. Log in as a user, escalate to root via sudo. This creates an audit trail.
• AllowUsers explicitly enumerates who can SSH in. Everyone else is denied even if they have valid credentials.
• Restrict SSH by source IP at the firewall level. Your SSH port shouldn't be reachable from the whole internet — only from your bastion, your VPN, or specific admin networks.

Better yet: Use SSM Session Manager (AWS), Azure Bastion, or Google IAP for server access. No inbound SSH port at all. Admins authenticate through IAM, and the session is logged.

2. Automatic Security Patching

Most breaches exploit vulnerabilities that had patches available for months. Automatic patching for security updates is the single highest-leverage server security control.

For Ubuntu/Debian:

apt install unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

Edit /etc/apt/apt.conf.d/50unattended-upgrades to enable security updates and optionally automatic reboot during a maintenance window.

For RHEL/Rocky/Alma:

dnf install dnf-automatic
systemctl enable --now dnf-automatic.timer

For Windows Server: Windows Update for Business via Intune or WSUS with reboot scheduling.

The concern teams raise: "What if a patch breaks something?" The answer: it happens occasionally, and the cost is manageable. The cost of NOT patching is an eventual breach. Automatic security patching with a reboot window wins the tradeoff.

For production-critical servers, test security patches in staging with automatic application, and production with manual review on a 7-to-14-day delay. Anything older than 14 days behind security updates is a finding.

3. fail2ban or Equivalent for Brute-Force Protection

Even with key-only SSH, the auth log fills up with failed attempts. fail2ban bans IPs that exceed a failure threshold.

apt install fail2ban
# /etc/fail2ban/jail.local
[sshd]
enabled = true
maxretry = 3
bantime = 1h
findtime = 10m

This is belt-and-suspenders with key-only auth, but it cuts log noise dramatically and catches scanning campaigns early.

For web servers, extend fail2ban to watch nginx/apache logs for brute-force attempts on login endpoints, WordPress wp-login.php, and other common targets.

Cloud-native alternatives: CloudFlare WAF, AWS WAF rate-based rules, Azure Front Door rate limiting. These work at the edge before requests reach your servers.

4. Audit Logging That Survives Compromise

Default logs live on the server. A compromise that gets root can edit or delete them. Logs that matter need to ship off the box in real time.

What to ship:

• /var/log/auth.log or /var/log/secure — authentication events
• /var/log/audit/audit.log — Linux auditd events
• Application logs from services that touch sensitive data
• Command history for admin users (via auditd or shell instrumentation)

Where to ship:

• Centralized syslog server on a separate network
• Cloud-native logging service (CloudWatch Logs, Azure Monitor Log Analytics, Google Cloud Logging)
• SIEM (Splunk, Sentinel, Elastic, Wazuh) for active analysis

Critical detail: The shipping configuration must not be writable by the processes being monitored. If a compromised web server can edit its own logging configuration, the logs are untrustworthy.

Run auditd with a minimal useful ruleset: track authentication, privilege escalation, file modifications in /etc, and changes to the audit configuration itself.

5. Immutable Infrastructure Where You Can

The oldest server security advice is "patch everything." The newer, better advice is "replace everything." An immutable infrastructure approach says you never SSH into a server to make changes — you build a new image, deploy it, and destroy the old one.

Benefits:

• No configuration drift. Every server matches the image.
• No long-lived backdoors. Servers that get replaced every 30 days limit attacker persistence.
• Patches get applied by building new images, not by mutating running systems.
• Golden images can be scanned and signed before deployment.

Tools:

• Packer for building images (AMIs, Azure Images, GCP Images)
• Terraform or Pulumi for deploying from those images
• Autoscaling groups or instance groups for rolling replacement

Not every workload can be immutable — stateful services, databases, long-running sessions. But the application tier almost always can be.

6. Host-Based Firewalls, Not Just Cloud Network Rules

Security groups and NSGs are necessary but not sufficient. A host-based firewall adds a second layer and catches things the cloud rules miss.

ufw on Ubuntu:

ufw default deny incoming
ufw default allow outgoing
ufw allow from <bastion-ip> to any port 22
ufw allow 443/tcp
ufw enable

firewalld on RHEL/Rocky:

firewall-cmd --set-default-zone=public
firewall-cmd --add-service=https --permanent
firewall-cmd --reload

Why two layers:

• Cloud network rules can be changed by anyone with cloud IAM. Host firewall rules require server access.
• Cloud network rules miss traffic between workloads on the same host.
• If the cloud control plane is compromised, host firewalls are the remaining protection.

Bonus: File Integrity Monitoring on Critical Paths

For servers that handle regulated data or critical operations, file integrity monitoring (AIDE, Tripwire, OSSEC, Wazuh) tells you when the system binary or config layout changes. This catches rootkits, backdoors, and configuration tampering.

Scope it narrowly — watching every file generates noise. Watch /etc, /bin, /sbin, /usr/bin, /usr/sbin, and any application binaries. Alert on unexpected changes.

What We'd Actually Do

For a new Linux server going into production:

1. Disable password SSH, root login, and restrict by source IP
2. Enable unattended-upgrades for security updates
3. Install and configure fail2ban for SSH and web endpoints
4. Configure auditd and ship logs off-host
5. Enable ufw with explicit allow rules
6. Apply CIS benchmarks where relevant using Ansible/OpenSCAP

Automate all of this with Ansible or a base image. Doing it manually on each server guarantees drift and missed steps.

Three Takeaways

1. Automatic security patching is non-negotiable. The "what if it breaks something" concern is much cheaper than an unpatched CVE being exploited.
2. Logs that live on the compromised server are not logs. Ship them off-host immediately.
3. Host-based firewalls and cloud network rules are both required. Defense in depth, not either/or.