Free Fixes First: Tuning TCP for Long-Distance Transfers

Somewhere between "the overseas transfer is too slow" and "sign the purchase order for an accelerated-transfer product" there is an afternoon of free configuration work that most organizations skip. That is a shame, because on the majority of long-haul links, the gap between the throughput you get and the throughput you paid for is not caused by anything a product needs to fix. It is caused by defaults — buffer limits sized for a local network, applications quietly capping their own windows, and connections that die of neglect three hours into a four-hour job.

This is the tutorial for that afternoon. You will raise the operating-system limits that keep TCP's window small, check the application layer for hidden caps, keep long-lived sessions alive through stateful firewalls, use parallelism where it honestly helps, and — the quiet champion — arrange to move fewer bytes in the first place. Every change is paired with a measure-before-and-after method, so you keep what works, revert what does not, and finish with numbers instead of impressions.

This article is part of our Accelerated Transfer series. The physics behind every fix — the bandwidth-delay product, or BDP — is worked through in why TCP slows down over distance; the one-line recap is that a TCP connection cannot exceed its window divided by the round-trip time, so on long paths, small windows mean slow transfers no matter how fat the link is.

What Tuning Can and Cannot Fix

Honest expectations first. Tuning raises the window — the amount of data TCP allows in flight while it waits for acknowledgments. That directly attacks the most common cause of slow long-haul transfers: a window far smaller than the link's BDP. If your link is long but clean (essentially zero packet loss), tuning alone can take a single stream from single-digit megabits to a large fraction of line rate. That is the normal case, and it is why this article comes before any purchase conversation.

What tuning cannot do: it cannot repair a path that drops packets, and it cannot remove TCP's deep backoff when loss occurs at high latency — the one scenario where UDP-based acceleration genuinely earns its price. It also cannot help if your bottleneck is a slow disk, an overloaded CPU, or thousands of tiny files paying per-file overhead. If you have not yet identified which problem you actually have, run the measurement protocol in do you actually need acceleration? first — it takes an afternoon and stops you from tuning the wrong thing.

The Method: Baseline, Change One Thing, Measure Again

Tuning folklore is everywhere — settings pasted from forums, applied in batches, never verified. The antidote is a small, strict loop. Establish a baseline, change exactly one variable, re-run the identical tests, and keep or revert based on the numbers. Here is the whole method in copyable form:

# ---- BEFORE/AFTER TUNING METHOD ----

# 1. BASELINE - run each test 3 times, keep the median. Change nothing yet.
ping -c 30 far-end.example.com            # record average RTT
iperf3 -c far-end.example.com -t 30       # record Mbps and Retr count
# ...plus one timed real transfer of the same 5 GB test file
# with your actual tool (SFTP/FTPS client, sync job, etc.)

# 2. COMPUTE the window you need:
#    BDP = bandwidth x RTT
#    e.g. 500 Mbps x 0.150 s = 75,000,000 bits = ~9.4 MB
#    Set OS buffer maximums to at least 2x the BDP for headroom.

# 3. CHANGE ONE THING (one sysctl, one client setting - never a batch).

# 4. RE-RUN the same three tests at a similar time of day.
#    Better?  Keep it, log it, move to the next change.
#    Same or worse?  Revert it, log that too.

# 5. LOG every run: date, time, direction, setting changed, result.
#    The log is your proof later - and your rollback map.

The one-change-at-a-time rule feels slow and saves weeks. When two changes land together and throughput doubles, you do not know which one worked, which one did nothing, and which will bite you at the next OS update. When something regresses next quarter, the log tells you exactly what a healthy configuration looked like.

Fix 1: Let the Window Grow — Buffer Limits and Window Scaling

Modern TCP stacks already contain the machinery for big windows: window scaling (the protocol extension that lets windows exceed the historical 64 KB limit) and autotuning (the operating system growing a connection's buffers automatically as it observes the path). You rarely need to switch these on — they are on. What you need to do is raise the ceilings that stop them partway, because default maximums on many systems sit far below a long link's BDP.

Linux endpoints

Two kinds of settings matter: the global socket buffer maximums, and the per-connection triplets (minimum, default, maximum) that bound TCP autotuning. Sized here for links up to roughly a 64 MB BDP — comfortably beyond a 500 Mbps, 150 ms path's 9.4 MB:

# Raise the ceilings (immediate effect, lost at reboot):
sysctl -w net.core.rmem_max=67108864
sysctl -w net.core.wmem_max=67108864
sysctl -w net.ipv4.tcp_rmem="4096 131072 67108864"
sysctl -w net.ipv4.tcp_wmem="4096 131072 67108864"

# Make it permanent: the same four lines (without "sysctl -w")
# in /etc/sysctl.d/90-wan-tuning.conf, then:  sysctl --system

# Sanity checks (both should print 1 = enabled):
sysctl net.ipv4.tcp_window_scaling
sysctl net.ipv4.tcp_moderate_rcvbuf

Note what these do: they permit growth; they do not allocate 64 MB to every connection. Autotuning still sizes each socket to what its path needs, so your LAN traffic keeps its small buffers and only the long-haul flows grow. That is why raising the caps is safe on shared servers.

Windows endpoints

Windows has shipped receive-window autotuning enabled for a long time, and at its normal level it can grow windows into the tens of megabytes. The practical problem is that fleets accumulate history: an old group policy, a registry tweak from an ancient troubleshooting session, or a third-party "network optimizer" may have set autotuning to a restricted level, silently reinstating a tiny window. Check, and restore the default if needed:

netsh interface tcp show global
    (look for: Receive Window Auto-Tuning Level : normal)

# If it says disabled, restricted, or highlyrestricted:
netsh interface tcp set global autotuninglevel=normal

Because Windows server applications ride the same stack, anything serving files over TCP inherits the fix — a Sysax Multi Server machine serving SFTP or FTPS to a distant office, for example, benefits from a healthy autotuning level with no application changes at all.

The application layer: the cap nobody checks

Here is the step most tuning guides omit. An application can request a fixed socket buffer, and that request quietly disables autotuning for its connections — the OS ceilings stop mattering because the app never asks for more. Transfer tools with modest hard-coded buffers are common, and some SSH-based programs additionally enforce an internal flow-control window of their own, above TCP, which caps throughput identically. The symptoms match a BDP ceiling exactly; the fix lives in the tool, not the kernel. Practical moves: check your client and server documentation for buffer or window settings and raise them; prefer current builds of transfer tools, since several widely used ones gained larger or dynamic internal windows over time; and re-test after any tool upgrade. Our SFTP series covers the SSH-specific version of this story in more depth.

Fix 2: Keepalives — Finishing the Transfer You Started

This fix raises no benchmark number, and it may still save your nightly job. Between your endpoints sit stateful devices — firewalls, NAT gateways, load balancers — that remember each connection and quietly forget any that look idle. Long transfers produce exactly that look: an FTP or FTPS session's control connection sits silent for hours while the data connection does all the work, and when the firewall expires the "idle" control session, the transfer dies at the 97% mark. The fix is keepalives — tiny periodic packets whose only job is to keep the state table warm:

• SSH-based transfers (SFTP): set ServerAliveInterval 60 in the client's SSH configuration (or the equivalent "send keepalive every N seconds" option in graphical clients).
• Operating-system level: on Linux, sysctl -w net.ipv4.tcp_keepalive_time=300 starts keepalive probes after five minutes of idleness for applications that enable them.
• Transfer software: most servers and clients expose a keepalive or NOOP-interval setting for exactly this purpose; a scheduled job that retries on failure helps too, but not dying beats retrying.

Think of keepalives as throughput insurance: a four-hour transfer that completes beats a faster one that dies at hour three and starts over. If your long jobs fail intermittently near the end, fix this before touching any performance setting, or your before/after measurements will be noise.

Fix 3: Parallel Streams — Multiply the Windows You Have

If one TCP stream carries one window per round trip, four streams carry four. Parallelism is the oldest trick for filling a long pipe, it works with completely stock software — run several transfers at once, or use a client that opens multiple connections — and it stacks with the buffer tuning above. It is also the fix with the sharpest edges: streams compete with other traffic more aggressively (four flows take four shares of a congested link), returns diminish quickly past a handful of streams, and on lossy paths each stream still suffers TCP's backoff individually. Use it as a complement to tuning, not a substitute. The honest full story — when parallel wins, when a single tuned stream wins, and how to pick a stream count — is the subject of its own article: parallel streams: what multi-connection transfer fixes and what it doesn't.

Fix 4: Move Less Data — The Speedup Nobody Benchmarks

Every fix so far makes bytes travel faster. The remaining family makes there be fewer bytes, and on repetitive transfers it beats everything else in this article combined:

• Compress before or during transfer. Text, logs, exports, and databases often shrink 3× to 10×. Compressing a 100 GB dataset to 25 GB is worth more than any window setting — it is a 4× speedup at any link speed. (Already-compressed media gains little; know your data.)
• Send only what changed. Delta-transfer techniques compare source and destination and ship just the differing blocks — routinely reducing a nightly "full" sync to a few percent of its size. Our rsync and delta transfer series is the deep dive.
• Batch small files into archives. Ten thousand tiny files pay ten thousand rounds of per-file overhead; one archive pays once and lets the window machinery actually spin up.
• Schedule into quiet hours. Off-peak, your transfer competes with nothing and congestion-driven loss drops. An automation tool such as Sysax FTP Automation can run the job at 2 a.m., zip the folder as a pre-processing step, and retry on failure — deadline met without a single protocol heroic.

A Worked Before/After: One Link's Story

Here is how the fixes compound on the running example — a clean 500 Mbps intercontinental link, 150 ms RTT, moving a 100 GB dataset. The times fall out of the arithmetic (100 GB = 800,000 megabits):

From sixty-three hours to sixteen minutes, and the invoice total is zero. The numbers are idealized — real links wobble, protocol overhead shaves a little, compression ratios vary — but the shape is exactly what well-run tuning efforts see on clean high-BDP paths: the first buffer fix delivers a huge multiple, the application-layer fix delivers the next one, and data reduction quietly doubles whatever remains.

When Tuning Is Not Enough

Play fair with yourself about the endgame. If your measurements show real, persistent packet loss on a long path — the compounding condition explained in the BDP article — tuned windows will not save a single TCP stream, because the constraint is congestion backoff, not buffer space. Parallel streams soften that case; they do not cure it. That is the honest boundary of this article, and the point where evaluating a UDP-based product becomes rational rather than reflexive. Cross it with your baseline log in hand: the fair comparison for any vendor demo is your tuned configuration, and you now have one, documented run by run.

Most readers never reach that boundary. Their links are long but clean, their windows were the whole problem, and after this afternoon their transfers run at line rate over the same SFTP and FTPS infrastructure they already trust. If that is where you land, finish the job like an engineer: write the final settings into configuration management, note the log's location in the runbook, and re-run the three baseline tests after major OS or tool upgrades so a silent regression never costs you another weekend.

The Version to Tell a Colleague

Before anyone buys transfer acceleration, spend one afternoon: raise the OS buffer ceilings so TCP's window can reach the link's bandwidth-delay product, check the transfer tool for its own hidden buffer cap, turn on keepalives so long jobs stop dying through stateful firewalls, add a few parallel streams if one tuned stream still falls short, and shrink the data with compression and delta transfer. Measure before and after every single change, one change at a time. On a clean long link this routinely turns days into minutes for free — and if loss, not windows, turns out to be the enemy, you will know it from your own numbers and can evaluate acceleration from a position of strength.

Related reading in this series: the parallel streams deep dive and the measurement protocol this article's method builds on.

Frequently Asked Questions