Choosing Protocols and Settings for Multi-Gigabyte Moves

For a small file, the protocol barely matters: SFTP, FTPS, HTTPS, or rsync will all move a five-megabyte report in a second. The choice comes down to what the far end supports. For a 40 GB file the same four protocols behave very differently. One of them cannot fill a long-distance link with a single connection unless you change a setting. One of them holds a second connection open and silent for half an hour, which some firewalls will not tolerate. One of them resumes uploads only if the far end was built for it. And all of them can be stopped at the last moment by a filesystem limit or a full disk that nobody checked.

This article compares the four for large files specifically — not their security, not their history, just how they behave when the transfer runs for a long time. Then it walks through the settings that matter: buffers and windows explained in plain words, and the cost of encryption on the CPU. It also covers the disk and filesystem limits that end transfers at 99 percent. It is part of our Large File Strategies series. If your question is which protocol is safer or which a partner should standardize on, see SFTP vs FTPS. Here we assume the file is big and the link is the problem.

What a Big File Asks of a Protocol

A large transfer puts four questions to a protocol that a small one never raises:

• Can it resume? Above some size, an interruption is a matter of time, and a protocol that cannot continue from where it stopped forces a restart from zero. The mechanics per protocol are in our resume and checkpoint restart series; here it is one line in the comparison.
• Can one connection fill the link? On a long-distance path, a single connection is limited by how much data can be in flight at once. Some protocols add their own limit on top of the network's, and the transfer crawls on a link that is mostly idle.
• What does encryption cost? Every byte of a 40 GB file is encrypted and decrypted. On modern hardware that is nearly free; on an old or small machine it can be the bottleneck.
• How many connections, and how long are they quiet? A protocol that uses one connection for everything is easy on firewalls and NAT. One that opens a second connection and leaves the first silent for the duration is exposed to every idle timer in the path.

The Four Candidates

SFTP

SFTP runs over an SSH connection: one TCP connection, always encrypted, carrying both commands and data. That single connection is its great operational strength for big files. Nothing goes quiet. There is no second data channel for a NAT device to forget, and one open port is all a firewall needs. It resumes uploads and downloads by writing or reading at an offset, and nearly every server allows it.

Its weakness is throughput on long paths. SFTP moves data as a series of requests, each carrying a block of the file, and the client keeps only a limited number in flight. On a local network that limit is invisible; across a continent it can cap a single transfer well below the link speed. The fix is a setting, covered below; the deeper story is in SFTP performance.

FTPS

FTPS is FTP with TLS encryption. It uses two connections: a control connection for commands and a separate data connection for the file. The data connection is a plain TCP stream with TLS on top, which means it has no per-request limit of its own. A single FTPS transfer fills a long link about as well as TCP allows, and the CPU cost is only the encryption. Resume is supported for both directions on most servers.

Its big-file weakness is the quiet control connection. While 40 GB moves on the data connection, the control connection sends nothing for thirty minutes. NAT devices and firewalls that forget idle connections drop it. After that, the client cannot send the command that ends the transfer, and the job reports failure for a file that arrived. Control-connection keepalives and NAT timers set for long flows are the fixes; see the timeouts and keepalives series and FTPS through firewalls and NAT.

HTTPS

HTTPS is the web's protocol, and for downloads of big files it is excellent: one connection, plain TCP with TLS. Downloads are resumable through byte-range requests that every serious client supports. No client software is needed beyond a browser or curl. For uploads it depends entirely on the server. A plain web form upload is a single request that must complete in one go. It is subject to request-size limits on the server and buffering in any proxy in between, and it cannot resume. Servers built for large uploads offer chunked or resumable upload schemes that fix all of that — but only if the far end runs one. The large files over HTTP article explains both sides.

The practical rule: HTTPS is a fine choice when the big file flows from a server to many recipients. It is also a fine choice when a person without a transfer client needs a one-off download. For a scheduled 40 GB upload, prefer SFTP or FTPS unless the receiving server specifically supports resumable uploads.

rsync

rsync, usually run over SSH, is famous for sending only the differences between two versions of a file. For a big file that does not yet exist on the far end that talent is irrelevant — the first copy sends everything. But rsync remains a strong big-file tool: it resumes with its partial-file options and verifies a whole-file checksum at the end. It handles temporary names and atomic renames itself. One cost is that both ends need rsync and SSH access. Its delta calculation reads the whole file on both sides when a previous version exists. And its compression option hurts on fast links. When a large file is re-sent regularly with small changes — a growing archive, a nightly database file — the delta transfer becomes the whole point. Our article on rsync over WAN covers that case.

Side by Side

On the receiving side, a server that speaks several of these lets the choice be made per partner rather than per server. Sysax Multi Server, for instance, serves SFTP, FTPS, and HTTPS from one Windows host. So a scheduled 40 GB feed can arrive over SFTP while an occasional human recipient fetches a large file through a browser over HTTPS.

Buffers and Windows in Plain Words

Every settings discussion about big transfers mentions windows, buffers, and in-flight data, so here is the idea in one paragraph. A sender cannot fire bytes into the network without limit; it sends a certain amount, then waits for the receiver to confirm arrival before sending more. The amount it may have outstanding — sent but not yet confirmed — is the window. The confirmation takes one round trip to come back. So the fastest a single connection can go is the window divided by the round-trip time, regardless of link speed. On a local network with a one-millisecond round trip, almost any window is enough. Across a continent at 80 milliseconds, it is not.

Put numbers on the running example: a 200 megabit link needs 25 MB per second, and 25 MB per second times 0.08 seconds is 2 MB. That is the amount that must be in flight at every moment to keep the link full — the bandwidth-delay product, explained fully in our article of that name. Modern operating systems grow the TCP window automatically to that size and beyond. So for FTPS and HTTPS the network usually takes care of itself, and the checks in TCP tuning first confirm it.

SFTP is the exception, because it adds a second window of its own. The client sends the file as numbered write requests of a fixed size and keeps a fixed number of them outstanding. In the OpenSSH client the defaults are 32 KB per request and 64 requests in flight, which is 2 MB. That is exactly the bandwidth-delay product of our example, with no margin. Double the distance to a 160-millisecond round trip and the same defaults cap the transfer at about half the link speed while the network sits idle. The fix is to raise the request count, the request size, or both:

# OpenSSH sftp: -B bytes per request, -R requests in flight
# 64 KB x 128 = 8 MB outstanding, enough for 200 Mbit at a 300 ms round trip
$ sftp -B 65536 -R 128 alex@10.20.30.40
sftp> put /srv/export/export.bin /inbound/export.bin.part
Uploading /srv/export/export.bin to /inbound/export.bin.part
export.bin                               42%   16GB  23.1MB/s   16:44 ETA

The progress line is the diagnostic: 23.1 MB per second on a 200 megabit link is the link running full. If the same transfer shows 9 MB per second on an otherwise empty link with a long round trip, the window is the bottleneck. In that case, these flags are the first thing to change. Larger values cost only a few megabytes of memory, so there is little reason to run the defaults on a WAN. Graphical and scripted clients have equivalent settings under names like "buffer size," "pipeline depth," or "parallel requests." When no window setting is enough because the path is very long, several connections in parallel each bring their own window. This is the parallel streams approach, and chunking the file is the simplest way to use it.

Ciphers and the CPU

Encrypting 40 GB is real work, and where it lands decides whether it matters. On any processor with hardware AES instructions, AES encryption runs at a gigabyte per second or more per core. Those processors include nearly every modern server and desktop CPU. That speed is far above any WAN link, and the cost is invisible. On small virtual machines, older hardware, or embedded devices without those instructions, AES falls to a hundred or two hundred megabytes per second. On a fast local link with such hardware, the transfer becomes CPU-bound. The quick way to know which world you are in is the benchmark that ships with OpenSSL:

$ openssl speed -evp aes-128-gcm 2>/dev/null | tail -1
aes-128-gcm     612345.67k  1893201.15k  3402188.80k  4188011.52k  4512877.23k  4530032.64k

The columns are throughput at increasing block sizes, in kilobytes per second; the rightmost figures are what a file transfer sees. Here the machine encrypts at roughly 4.5 GB per second, so encryption will never be the bottleneck. If the numbers come out around 150,000k instead, the CPU lacks acceleration, and two changes help. In that case, choose a cipher designed for software, such as ChaCha20, which is often two to three times faster than AES on such hardware. Also make sure session compression is off, because it burns the same scarce CPU. In an OpenSSH configuration that is one line, Ciphers chacha20-poly1305@openssh.com,aes128-gcm@openssh.com, listing the preferred cipher first. The security side of choosing ciphers is in cipher policy basics. Note that the same choice has to be made on both ends of a transfer, and the slower of the two machines sets the pace.

The Limits That Stop Transfers at the Last Moment

A large transfer that fails at 99 percent is nearly always stopped by a limit on the receiving machine, not by the network. These are the ones to check before starting, in the order they usually bite:

• Filesystem file-size limits. A FAT32 volume cannot hold a file over 4 GB. It fails with a misleading "disk full" or "invalid argument" when the file reaches that size. Some network shares and older appliances hide the same limit. Check the destination filesystem type; NTFS, exFAT, ext4, XFS, and similar are fine.
• Free space at the peak. The destination needs room for the file, plus any temporary copy the server or a post-processing step makes, plus a margin. A temporary folder on a different volume from the final folder turns the finishing "rename" into a full second copy that needs the space twice. Check with df -h on Linux or Get-Volume in PowerShell, and check the temporary location separately.
• Account quotas. The volume can have terabytes free while the receiving account is capped at 50 GB. The failure message is often just "write error." Ask the far end's administrator, or test with a file of the intended size.
• Server-side maximum file size. Some transfer servers and web servers enforce their own per-file or per-request limit independent of the disk. It is a configuration setting, and it is rarely mentioned until it is hit.
• Post-transfer processing time. When the last byte lands, the server may scan the file for malware, flush it to disk, or move it. Only then does it send the final reply. On a 40 GB file that can take minutes, and a client that waits sixty seconds for the reply reports failure. Raise the client's response timeout for big files; the timeouts series covers the settings on each side.
• Session age limits. A firewall or load balancer with a maximum connection lifetime kills a long transfer at the same elapsed time every run. If a transfer always dies at, say, sixty minutes, that is the cause; the remedy is chunking below the limit or a change to the device.

Every one of these is knowable in advance and cheap to check. None is visible in the transfer log until the transfer dies. That is why the pre-flight checklist in what changes when files get large puts them ahead of any protocol tuning.

A Settings Checklist for One Big Move

The decisions above, condensed into the order to make them:

1. Protocol. Scheduled upload: SFTP by default, FTPS where the partner is standardized on it, rsync when the same file is re-sent with changes. Download to many or to a person: HTTPS. Upload over HTTPS only if the server has a resumable upload scheme.
2. Resume enabled and tested. Client flag set, server permission confirmed, tested with a deliberately interrupted small file — then designed around temporary names and a state file as in designing large transfers to resume.
3. Window sized for the distance. Measure the round-trip time, multiply by the link speed in bytes per second, and make sure at least that much can be in flight. For SFTP, raise the request count and size explicitly.
4. Cipher checked on both ends. Run the benchmark; on hardware without AES acceleration prefer ChaCha20 and disable session compression.
5. Compression decided. On for compressible data over slow links, off otherwise — the test and arithmetic are in compression for large transfers.
6. Receiving limits verified. Filesystem type, free space at the peak including temporary locations, account quota, server maximum file size, client response timeout, session age limit in the path.
7. Keepalives on for two-connection protocols. For FTPS, control-connection keepalives shorter than the smallest NAT or firewall idle timer in the path.

The Version to Keep in Your Head

For a big file, SFTP is the calm default — one connection, resumable, firewall-friendly — provided its request window is raised to match the distance. FTPS fills a long link without tuning but leaves a quiet control connection exposed to idle timers. HTTPS is superb for downloads and only as good for uploads as the server was built to be. rsync earns its place when the same large file is sent again and again with changes. Under all of them, the same rules apply. In-flight data must cover the bandwidth-delay product. Encryption is free on modern CPUs and a bottleneck on old ones. The limits that actually stop transfers live on the receiving disk, where a two-minute check prevents a thirty-minute loss.

From here, verifying large transfers without doubling the time covers proving the file arrived intact. Our article on protocol performance compared gives the measured throughput picture behind this article's rankings. When a transfer is slow and the reason is not obvious, the systematic approach is in the diagnosing slow transfers series.

Frequently Asked Questions