Verifying Your Hardening Actually Holds

Here is the uncomfortable truth about every hardening effort: the day you finish is the best the server will ever look. From that moment, entropy goes to work — a temporary firewall rule that stays, an update that resets a default, a test account created in a hurry and never removed. None of these announce themselves. Security loosening is silent by nature: when a port opens that shouldn't be open, nothing breaks, no one is paged, and the monitoring stays green. The only way to know your hardening still holds is to check.

Verification is that checking, done deliberately and on a schedule: looking at the server from the outside the way a stranger would, comparing its configuration to the baseline you wrote down, and re-running the tests that proved your controls worked the first time. It is the verify step of the baseline-verify-maintain loop from the hardening program overview, and it is the step most programs skip — which is why so many "hardened" servers are hardened only in the documentation. This closing article of our Server Hardening series makes the step concrete and small enough to sustain.

You will leave with three things: an external scanning routine, a drift-detection habit that costs minutes, and a self-audit checklist sized for a small team's calendar.

Why Hardened Servers Drift

Configuration drift is the gap that grows between a system's intended state and its actual state. Nobody plans it; it accumulates through perfectly ordinary events:

• Temporary changes that outlive their reason. A firewall opening for a vendor's remote session, a debug setting enabled during an incident, a partner's IP allowlisted "for this week." Each was legitimate; each needed an expiry date that nobody set.
• Updates that reset or add. Upgrades restore default greetings, re-enable components you removed, or introduce new features switched on out of the box. The patch made you safer in one dimension and drifted you in another — one more reason the patching runbook ends with a verification pass.
• People and process changes. A new administrator configures things the way their last shop did. A colleague creates a test account during troubleshooting. A server restored from backup comes back with settings from three months ago — including the ones you tightened since.
• The estate grows. A second transfer server appears for a project, cloned from an image made before your hardening pass. Drift is not just change on one box; it is new boxes that never got the treatment.

Notice the common thread: every cause is invisible from the inside during normal operation. Transfers keep working — often they work better, since loosening tends to remove friction. That is exactly why verification has to be an explicit activity rather than something you'll notice in passing. You won't.

The Outside View: Scan Your Own Server

The first verification habit is seeing your server the way the internet sees it. One standing rule before the commands: scan only systems you own or have explicit written permission to scan. Probing other people's machines is unauthorized activity in most jurisdictions; everything below is aimed at your own estate. Run the scan from a genuinely external vantage point — a home connection, a phone hotspot, a small cloud instance — because a scan from inside your LAN answers a different, easier question than the one that matters.

The workhorse is a port scan: a tool that tries to connect to a list of ports and reports which ones answer. The standard tool, nmap, runs on both Windows and Linux:

# Authorized scan of your own server, from an external network:
nmap -Pn -p 21,22,443,990,50000-50100,3389 files.example.com

PORT      STATE    SERVICE
21/tcp    open     ftp
22/tcp    open     ssh
443/tcp   open     https
990/tcp   closed   ftps
3389/tcp  open     ms-wbt-server   <-- not in the baseline. Investigate.

# Optional: -sV asks services to identify themselves, showing you
# what banner-level detail a stranger's fingerprinting would see.

Reading the result is a one-step algorithm: compare against the baseline's port list. Every open port is either on the list — fine — or it is a finding, like the remote-desktop port above appearing on an internet-facing interface. "Finding" does not automatically mean "attack"; it usually means a firewall rule drifted. But it means the outside world can see something you decided it shouldn't, and that gets investigated today, not at the next quarterly review.

Two extensions round out the outside view. First, banner and handshake detail: the -sV flag, plus the self-check commands from the information-leakage article, tell you whether trimmed greetings stayed trimmed. Second, encryption posture: for TLS-based services, verify from outside that old protocol versions stay refused and certificates are valid — the outside-in TLS checks are walked through in hardening FTPS. Public scan databases that continuously index the internet are worth a periodic query for your own addresses too: they show what has already been recorded about you, which is occasionally how teams discover an exposure that predates their tenure.

The Inside View: Catching Configuration Drift

External scanning sees what leaks out; drift detection sees what changed within. The practice is comparison against a recorded known-good state, and the lightweight version needs no products at all:

• Export and store the configuration after every sanctioned change: the server's settings export, the relevant OS config files, the firewall rules (Get-NetFirewallRule piped to a file on Windows, the ruleset listing on Linux), and the account list. Store the exports somewhere versioned — a proper version-control system if you have one, or at minimum a dated folder on a different machine. The point is that "what did this look like last month, and who changed it?" becomes answerable.
• Diff on a schedule. A scheduled task exports the same items fresh and compares: Compare-Object or fc on Windows, diff on Linux, file hashes (Get-FileHash / sha256sum) where whole-file comparison is simpler. An empty diff takes seconds to confirm. A non-empty diff is either a change you made on purpose last week — in which case the stored export was stale, itself a small process finding — or a change nobody claims, which is the interesting kind.
• Watch the crown-jewel files. The idea has a formal name — file integrity monitoring, tooling that alerts when designated files change — but the concept is just "hash the files that define the server's security posture, and alert on unexplained changes." Config files, the jail tree's top-level structure, the service binaries.

Then there are the checks that need to be functional rather than textual, because their failure mode is silence: does the canary transfer still run and alert when broken? Does a deliberately failed login still appear in the log — and does the log still ship off the box? Logging that quietly died is among the worst drift there is, because you discover it at exactly the moment you need the missing logs; the pipeline-watching practices in our transfer logging and audit series exist for that reason.

A worked example makes the habit concrete. Suppose your baseline folder holds firewall-rules.txt, accounts.txt, and the server's settings export, each saved after the last sanctioned change. A scheduled task runs weekly: it regenerates all three, compares them to the stored copies, and emails the result. Forty-nine weeks a year the mail says "no differences" and costs you five seconds. Then one week it shows a new allow rule named temp-vendor dated three weeks back. That mail just did the thing no amount of diligence-in-the-moment reliably does: it noticed a change that nothing else would ever have surfaced, while it was still just a finding and not yet a story.

What Drifts Most

Every estate is different, but the same suspects lead the lineup often enough to deserve named attention in your checks: accounts (test users, departed staff, partners whose contracts ended — each a live credential nobody is watching); firewall rules (the "temporary" opening is practically a genre); certificates and host keys (not weakened by change but by time — expiry is drift on a timer, and the monitoring habits in our certificate management series pair naturally with this article); encryption settings (an upgrade quietly re-enabling an old protocol version); and banners and error verbosity (restored to chatty defaults by updates, as covered in the leakage article). Your first few verification cycles will teach you your own top five; promote them to the front of the checklist.

One special case deserves a rule of its own: new servers verify before they serve. A box cloned from an image, restored from backup, or stood up quickly for a project inherits whatever state its source had — which may predate every hardening decision you have made since. Running the self-audit below against a server on its first day costs an hour and regularly catches the clone that came up with default banners, an open management port, and the image's original test account still enabled.

A Schedule a Small Team Can Sustain

Verification fails in practice for one reason: it is nobody's job on no particular day. The fix is a cadence, sized honestly for a small team, with the expensive checks rare and the cheap ones automated:

Put the monthly and quarterly entries on the actual team calendar with a named owner, and rotate the owner — rotation spreads the knowledge and keeps one person's blind spots from becoming the program's. The weekly skim gets dramatically cheaper when the server's own records are decent: an activity log like the one Sysax Multi Server keeps, with sessions and failures attributed per account, turns "grep the estate" into "read one report." And when the failed-login lines start telling a story — bursts, sprays, patient guessing — the response side of that story is our brute-force protection series.

The Self-Audit Checklist

This is the monthly hour, structured. It mirrors the five layers of the program's baseline checklist, but every line is a test with evidence, not a setting to admire — the difference between auditing and re-reading your own documentation:

HARDENING SELF-AUDIT — host: ______  date: ______  auditor: ______

1. OUTSIDE VIEW
   [ ] External port scan matches baseline list (attach output)
   [ ] Banners/greetings still trimmed; authorized-use notice present
   [ ] Old TLS/SSH protocol versions refused when offered from outside
   [ ] Certificate valid, correct names, expiry > 30 days out

2. OPERATING SYSTEM
   [ ] Listening-port inventory matches last saved inventory (diff)
   [ ] Patch level current per vendor advisories; last patch date noted
   [ ] Local firewall default-deny still active; rule diff reviewed
   [ ] Service workers run as the limited account (spot-check process list)
   [ ] Data volume separate; free space and quota headroom acceptable

3. ACCOUNTS AND JAILS
   [ ] Account list diffed against registry: every account has an owner,
       purpose, and future review date
   [ ] No shared accounts appeared; leavers and ended partners disabled
   [ ] Jail spot-test on 2-3 accounts: log in, attempt to escape and to
       list another account's folder — both fail
   [ ] Automation credentials: rotation dates within policy

4. SERVICE CONFIGURATION
   [ ] Config export diffed against stored known-good: empty, or every
       difference explained and baseline updated
   [ ] Only required protocols enabled; encryption floors unchanged

5. MONITORING PROVES ITSELF
   [ ] Deliberate failed login appears in the log within minutes
   [ ] Log shipping confirmed at the destination, not just the source
   [ ] Canary transfer job: last runs green; break it once, alert fires
   [ ] Config backup restore-tested this quarter (date: ______)

FINDINGS: ____________________  each with: fix, cause, baseline updated?

Adapt freely, but keep two properties. Evidence over assertion: "scan output attached" beats a checked box. And bounded size: if the audit stops fitting in an hour, split rarely-drifting lines into the quarterly pass rather than letting the monthly one die of thoroughness. The jail spot-test technique — logging in as a real account and attacking your own walls — comes from the provisioning checklist in account isolation and jails; run it against different accounts each month.

When a Check Fails

A finding is the system working, so treat it with curiosity instead of blame. Three questions, in order. What is the exposure? Fix the instance first — close the port, disable the account, restore the setting — at whatever urgency the exposure deserves. How did it happen? Trace the drift to its cause: a change that skipped the process, an update behavior nobody knew, a clone from a stale image. The cause is the real finding; the open port was just its symptom. Will it happen again? Repeated drift in the same spot earns automation — a scheduled script that re-asserts the setting, or an alert on that specific change — because anything that drifts twice will drift a third time. Then record the finding and its fix in the baseline's changelog. Over a year, that changelog becomes something valuable: proof, in your own records, that the program runs — which is precisely what an auditor, a customer questionnaire, or your own management means by "demonstrate your security process."

Close each finding by re-running the exact check that caught it, not by trusting the fix. If the scan found the port, the scan confirms the closure; if the jail test found the gap, the jail test proves the wall. It sounds pedantic until the first time a fix turns out to have been applied to the wrong rule, on the wrong profile, or on the wrong server — verification of the fix is just the loop applied at small scale, and it costs a minute.

Closing the Loop

This article ends the series where the first one began: hardening is a loop, not a project. The program overview set the baseline, the layer articles built it, the patching article maintains it, and verification is the pull that keeps reality attached to the plan. Run the loop at the cadence above and the server you hardened stays the server you have — not a fond memory with a firewall rule named "temp-vendor-access" quietly holding the door.

Frequently Asked Questions