Non-Interactive SFTP: Automation Without the 2 a.m. Password Prompt

Every administrator eventually inherits or builds the same thing: a scheduled job that moves files over SFTP while everyone sleeps. And every administrator eventually gets the same education: a job that works perfectly when you run it by hand hangs mysteriously at 2 a.m., because somewhere in the machinery a prompt appeared — a password request, a host-key confirmation, a passphrase — and there was no human at the keyboard to answer it. The job does not fail; it waits, silently, until a timeout or a Monday morning discovers it.

This cookbook is about building SFTP jobs that never wait for a human: batch mode that fails instead of asking, host-key management that is both prompt-free and actually secure (the popular shortcut is neither), credential handling that survives a security review, exit codes and verification your alerting can trust, and the scheduler realities — cron and Windows Task Scheduler alike — that break jobs in ways no manual test reveals. It is the operational payoff of our whole SFTP In Depth series: the protocol was practically designed for automation, and done right, unattended SFTP is boringly reliable.

The Enemy: Anything That Asks a Question

An interactive SFTP session may ask a human up to three things: "Are you sure you want to continue connecting?" (the host-key confirmation on first contact with a server), "Enter passphrase for key" (if the private key is passphrase-protected), and "Password:" (if the account authenticates by password). At a terminal these are three keystrokes. Under a scheduler they are three ways to hang, because the prompt is written to a console nobody is watching and the job blocks on input that will never come.

The design principle for everything that follows: a non-interactive job must have every question answered in advance — and must fail immediately, loudly, and with a nonzero exit code if a question comes up anyway. Fail-fast beats hang-forever in every operational dimension: the retry can happen sooner, the alert fires the same night, and the log says why.

Batch Mode: Fail, Don't Ask

The OpenSSH sftp client has both halves of the principle built in. The -b flag runs commands from a batch file instead of a keyboard, and — less obviously — it implies batch mode, in which the client refuses to prompt for anything: any would-be question becomes an immediate failure. Here is a complete, honest example — the batch file, and the wrapper script a scheduler actually invokes:

# ---- nightly-orders.batch : the SFTP commands ----
# a leading dash means "tolerate failure of this one command"
-mkdir /incoming/archive
cd /incoming
put /data/export/orders.csv orders.csv.part
rename orders.csv.part orders.csv
bye

# ---- nightly-orders.sh : the wrapper the scheduler runs ----
#!/bin/sh
LOG=/var/log/transfer/nightly-orders.log
sftp -b /etc/transfer/nightly-orders.batch \
     -i /etc/transfer/keys/orders_ed25519 \
     -o IdentitiesOnly=yes \
     -o UserKnownHostsFile=/etc/transfer/known_hosts \
     -o StrictHostKeyChecking=yes \
     -o ConnectTimeout=30 \
     orders@files.partner.example >> "$LOG" 2>&1
STATUS=$?
echo "nightly-orders exit=$STATUS" >> "$LOG"
exit $STATUS

Read the batch file's shape: it uploads to a temporary name and renames on completion — the atomic-delivery pattern from SFTP's file operations, which matters double for unattended jobs since nobody is watching for half-written files. The leading dash on mkdir tolerates "already exists"; every other command is strict, and the first failure aborts the batch with a nonzero exit. The wrapper pins down everything the environment might otherwise supply: which key (-i, with IdentitiesOnly so no other keys are offered), which known-hosts file, strict host checking, a connection timeout so an unreachable server fails in seconds rather than hanging, and a log that captures both output streams.

The Collection Job: Downloading Unattended, Safely

Half of all scheduled transfers run the other direction — polling a partner's server and collecting what has appeared. The batch-mode mechanics are identical (get instead of put), but collection jobs carry two extra design questions that upload jobs never face.

Question one: when is it safe to delete the remote file? The tempting sequence — download, delete, exit — has a failure mode where the download succeeds only partially, the delete succeeds completely, and the data now exists nowhere intact. The robust order is: download, verify locally (at minimum, compare the size against the remote listing), and only then delete the remote copy — or better, rename it into a remote archive/ folder if the server allows, so recovery from any mistake is a rename rather than a re-request to the partner. Because the verify step needs logic between SFTP commands, collection jobs usually run as two batch invocations (fetch, then clean up) or as a script driving the client twice.

Question two: what happens when the same file is seen twice? Partners re-upload corrected files; retries re-download after a crash mid-cleanup. A collection job must be idempotent — processing the same file twice must not double-post orders or overwrite good data with old. The usual disciplines: move collected files immediately into a dated local staging folder, feed downstream systems from there, and keep a small ledger (filename, size, checksum, collection time) the job consults before treating a file as new. Pair it with the partner using the temp-then-rename upload pattern on their side, and the "collected a half-uploaded file" class of incident disappears entirely.

Host Keys Without the Prompt — and Without the Foot-Gun

The first-connection prompt exists because the client has never seen this server's host key and wants a human to vouch for its fingerprint. Schedulers cannot vouch, so the internet's most-pasted "fix" is StrictHostKeyChecking=no — accept whatever key any server presents. Understand what that trades away: host-key checking is the mechanism that stops a redirected or intercepted connection from quietly succeeding against an imposter server. Disabling it makes your job deliver files — and credentials, if password authentication is in play — to whoever answers the address.

The seeding procedure, start to finish:

# 1. Fetch the server's public host keys
ssh-keyscan -t ed25519,rsa files.partner.example >> /etc/transfer/known_hosts

# 2. VERIFY before trusting: print fingerprints of what you just fetched...
ssh-keygen -lf /etc/transfer/known_hosts

# 3. ...and compare against the fingerprint the server's administrator
#    published through a separate channel (their docs, a signed email,
#    a phone call). Match: done. No match: stop and investigate.

Step 3 is the whole point — ssh-keyscan alone just trusts the network on a different day. Verify out-of-band once, and every future run verifies cryptographically for free. Two refinements finish the subject. Use a per-job known-hosts file (as the wrapper above does): the job stops depending on any user's home directory, which prevents the classic "works when I run it, fails from the scheduler" mystery — schedulers run jobs as other accounts, with other home directories, where the host key you accepted interactively does not exist. And when a partner announces a host-key change, update the seeded file through the same verify-first ritual; a job that suddenly fails with a host-key mismatch you did not expect is doing its job — treat it as an alarm, not an obstacle.

Credentials That Survive a Security Review

The authentication story for unattended jobs is short, because the authentication article carries the depth: use a dedicated service account with key authentication — one account per integration, one key per client machine, no passphrase (schedulers cannot type), compensated by tight file permissions and server-side restrictions (an sftp-only account confined to its own folder, per the server configuration guide). Rotation happens by adding the new key before removing the old, so there is never a night the job cannot log in.

What deserves equal attention is the leak catalog — the ways credentials escape from automation, all preventable:

• Passwords on command lines. Anything in a command line is visible to every local user in the process listing for as long as the command runs. No secret belongs there, ever.
• Secrets pasted into scripts. Scripts get copied, committed, emailed, and backed up. A password in script text will eventually be somewhere you never intended. Keys kept as separate, permission-locked files (or credentials in a restricted store) do not travel with the script.
• Keys in version control. Commit the wrapper and the batch file; never the private key. A key that has ever been pushed to a shared repository is compromised — rotate it.
• Password-feeder wrappers. Tools that stuff a stored password into an interactive prompt keep password authentication alive with extra fragility on top. They are a signal that the integration should move to keys.
• Chatty logs. Log command outcomes, not command lines with secrets, and rotate logs with sane permissions — transfer logs reveal filenames and business rhythms even when they hold no credentials.

Exit Codes and Verification: What a Script Can Actually Trust

The contract from batch mode is deliberately blunt: exit 0 means every command succeeded; nonzero means the run failed — whether because a transfer errored, the batch aborted mid-list, or the connection itself never came up. Resist the temptation to parse deeper meaning out of the number; the durable pattern treats any nonzero as "failed, alert, retry later" and mines the log text for the why (connection refused, host-key mismatch, permission denied — each writes an unmistakable line).

Exit codes only report on runs that end, so give every job a way to end. ConnectTimeout (in the wrapper above) bounds the connection attempt; for hangs after the session is up — a network path that dies mid-transfer and leaves TCP waiting patiently — add keepalive probing with -o ServerAliveInterval=30 -o ServerAliveCountMax=4, which abandons a dead peer after roughly two silent minutes instead of never. A job that cannot hang is a job whose failures are always visible to the scheduler.

But a zero exit answers only "did the protocol conversation succeed?" Production jobs verify the outcome, cheaply, right after: does the delivered file exist under its final name, and does its size match the source? For high-stakes feeds, exchange a checksum sidecar file and compare. And instrument both directions: alert on failure, but also alert on silence — a heartbeat line per successful run, monitored for absence, catches the failure modes that produce no error at all (the scheduler that quietly stopped scheduling, the job disabled during a maintenance window and never re-enabled). A job that has not reported success by 6 a.m. is a page-worthy event even if nothing reported failure.

Scheduler Realities: Where Manual Tests Lie

The final class of surprises comes from the scheduler itself, because a scheduled run's environment is not your terminal's:

The test that catches most of these before the first bad night: run the job via the scheduler, as the production account, with your own session logged out — not just by hand from your terminal. The scheduler's environment is the one that counts.

When the Script Grows Up: Purpose-Built Automation

Everything above can be assembled from a shell script, and for one simple job it is a fine craft. But watch what the script accrues over a year: retry logic, alerting, a lock file, log rotation, a second partner, a folder to monitor for arriving files, an encryption step because a counterparty requires it, sequencing because job B must follow job A. Each is a solved problem; solving them all again in every script is how organizations end up with a drawer of fragile, subtly different transfer scripts nobody dares touch — a pattern common enough that our FTP protocol series devotes an article to inheriting them.

This is the point of a dedicated automation tool. Sysax FTP Automation packages the patterns of this article as configuration rather than code: scheduled and scripted transfers over SFTP (plus FTP and FTPS), folder monitoring that triggers a job when files appear, built-in retry and error handling, OpenPGP encryption and decryption of payloads, and pre- and post-processing steps — which is where the temp-then-rename and verification stages naturally live. The judgment call is honest and simple: one stable job, script away; a growing portfolio of unattended transfers with alerting and audit expectations, and the purpose-built tool earns its keep quickly. Whichever route you take, the server side of the story — locked-down, sftp-only service accounts — is equally load-bearing, and it is the subject of the server configuration guide.

The Version to Tell a Colleague

Unattended SFTP fails through prompts, so eliminate every question in advance and make the rest fail fast: batch mode with a batch file, a dedicated key (no passphrase, tightly permissioned, offered explicitly), a pre-seeded and out-of-band-verified known-hosts file with strict checking kept on, and a connection timeout. Treat any nonzero exit as failure, verify the delivered file's existence and size, and alert on silence as well as on errors. Then test under the real scheduler and account, because PATH, home directories, and boot timing are where manual tests lie. And when one script becomes a portfolio, move the portfolio to purpose-built automation before it becomes folklore.

Round out the series with SFTP authentication for the key-management depth, file operations and their quirks for delivery patterns that survive any server, and SFTP performance to make the nightly window comfortable.

Frequently Asked Questions