How UDP-Based Accelerated Transfer Protocols Work

Sooner or later a vendor demo lands on your desk: the same large file, sent over the same intercontinental link, arrives twenty times faster with their product than with your SFTP job. The demo is usually real. The question a careful administrator asks next is why — because until you know what the product actually does under the hood, you cannot tell whether the speedup comes from genuine engineering, from your own untuned baseline, or from a bit of both.

This article opens the hood. Commercial accelerated-transfer products differ in branding and pricing, but nearly all of them are built from the same four ideas: send data over UDP instead of TCP, rebuild reliability at the application layer, replace TCP's window-based congestion control with a rate-based scheme, and recover from packet loss without stalling the flow. By the end you will understand each idea well enough to ask sharp questions in an evaluation — and to recognize which claims on the datasheet are physics and which are marketing.

This is part of our Accelerated Transfer series. It assumes you have met the bandwidth-delay product; if not, start with why TCP slows down over distance, because acceleration only makes sense as an answer to that specific problem.

The Problem Being Solved, in One Paragraph

TCP moves at most one window of data per round-trip time (RTT), so on a long path its throughput is capped at window ÷ RTT regardless of link speed. Worse, TCP reads every lost packet as congestion, cuts its window sharply, and rebuilds it one cautious increment per round trip — so on a 150 ms path with even 0.1% loss, a single stream can be stuck in the low single-digit megabits on a half-gigabit link. Both behaviors are deliberate design choices inside TCP, and neither can be fully disabled from an application. If you want fundamentally different behavior, you have to get out from under TCP entirely. That is the entire reason accelerated protocols exist.

What UDP Actually Is (and Why It Is Not "Faster")

UDP — the User Datagram Protocol — is TCP's minimalist sibling. It gives an application exactly one service: it takes a chunk of data, called a datagram, wraps it with source and destination port numbers and a basic checksum, and sends it. That is the whole protocol. No connection setup, no acknowledgments, no retransmission, no ordering, no flow control, no congestion control. Datagrams may arrive out of order, duplicated, or not at all, and UDP will neither notice nor care.

An analogy that keeps the two straight: TCP is registered mail with return receipts and a strict dispatch policy — dependable, but the sender constantly waits for confirmations. UDP is dropping postcards into a mailbox: cheap and instant, with no idea whether any of them arrive.

Two consequences matter for our topic. First, UDP is not faster than TCP on the wire. Bits are bits; a UDP datagram and a TCP segment cross the same ocean at the same speed. Anyone who tells you "UDP is a faster protocol" is compressing the truth past the breaking point. Second, UDP's emptiness is precisely its value here: because it imposes no sending policy at all, a protocol designer can build a different policy on top — one designed for long, fat, slightly lossy pipes instead of for the general internet. UDP is not the speedup. UDP is the blank sheet of paper the speedup is written on.

Proof that UDP alone solves nothing: TFTP, the tiny bootstrapping protocol, also runs over UDP — and because it naively sends one block and waits for each acknowledgment, it is one of the slowest transfer protocols in existence. (Our TFTP series covers it.) The intelligence, not the datagram, is what accelerates.

Rebuilding Reliability Above UDP

File transfer has a non-negotiable requirement: every byte must arrive, intact and in order, or the transfer must fail loudly. TCP provided that guarantee; leave TCP behind and you must rebuild it yourself. Accelerated protocols do so with machinery that will look familiar, because it is TCP's machinery relocated one layer up — where the designer, not the operating system, sets the policy.

• Sequence numbers. The sender stamps every datagram with its position in the file (or in the stream). The receiver uses the stamps to reassemble data in the correct order, whatever order the network delivered it in.
• Gap tracking and negative acknowledgments. The receiver keeps a map of which ranges have arrived. Instead of confirming every packet the way TCP does, it periodically reports the holes: "I am missing 40,001 through 40,120." This is a NACK — a negative acknowledgment — and it is far more economical on a long path, because a clean transfer generates almost no reverse traffic and a lossy one generates exactly the repair requests needed.
• Selective retransmission. The sender keeps recent data buffered and resends precisely the reported ranges — without pausing the main flow. New data keeps streaming while repairs slot in behind it.
• Integrity checking. Checksums or cryptographic hashes verify blocks and usually the whole file, so corruption is caught even if every datagram technically "arrived." Done well, this equals or exceeds what TCP's own modest checksum provides.

The result is a delivery guarantee equivalent to TCP's. When a security or operations colleague asks "doesn't UDP mean unreliable?", the accurate answer is: raw UDP, yes; these protocols, no — reliability has been rebuilt above it, and the file either arrives verified or the transfer errors out.

Rate-Based vs Window-Based: The Heart of the Matter

Reliability is table stakes. The actual performance difference lives in congestion control — the rules deciding how fast to send.

TCP's scheme is window-based. The sender may have one window of unacknowledged data outstanding; the window inflates gradually and collapses on loss; and because new budget only arrives with acknowledgments, the sending rate is chained to the round-trip time. Long RTT means slow feedback, slow window growth, and slow recovery — the whole pathology from the BDP article.

Accelerated protocols use rate-based sending instead. The sender picks a target rate — say 450 Mbps on a 500 Mbps path — and an internal timer releases datagrams at that pace, steadily, like a metronome. Acknowledgment traffic still flows back (for reliability and for feedback), but the sending rate is not coupled to it. The RTT can be 1 ms or 300 ms; the metronome ticks the same either way. Feedback from the receiver — measured loss, growing queueing delay — adjusts the rate smoothly: a little loss trims the rate slightly and triggers precise retransmissions; it does not halve anything. Distance stops mattering, which is precisely the property you want on a long pipe, and loss stops being a catastrophe, which is what you want on a slightly dirty one.

The diagram below contrasts the two behaviors on the same lossy long-haul link: TCP's sawtooth — climb, collapse on each loss event, climb again at round-trip pace — versus a rate-based sender holding near line rate and dipping only slightly to repair.

One honest caveat belongs right next to that flattering picture. A metronome that ignores feedback is not a protocol; it is a flood. Rate-based senders must still respond to congestion signals, and reputable implementations do — they measure loss and delay and back the rate down when the path is genuinely full, and most let the administrator set a hard ceiling. But the discipline is chosen by the vendor, not enforced by decades-old operating-system code, and an aggressively configured accelerated flow can crowd out every TCP stream sharing the path. That is not a reason to avoid the technology; it is a reason to configure rate caps deliberately and to expect your network team to ask about it — which they will, as covered in getting UDP transfer flows through security review.

Loss Recovery: Resend, Pre-Repair, or Both

Even with a friendly sending rate, long paths lose packets, so every accelerated protocol needs a repair strategy. There are two families, and many products blend them.

Selective retransmission is the NACK mechanism described earlier: the receiver reports holes, the sender refills exactly those holes, the flow never stops. Its cost is one round trip per repair — the missing range cannot be filled sooner than the time it takes the NACK to travel back and the resent data to travel out. For bulk file transfer this is usually fine, because the repaired block simply slots into the file while later blocks keep arriving.

Forward error correction (FEC) attacks the round trip itself. The sender transmits extra repair packets — mathematically derived redundancy, conceptually similar to parity in RAID — so the receiver can reconstruct a small number of missing datagrams from the extras it already holds, without asking for anything. Loss gets healed in zero round trips, at the price of overhead: five percent repair traffic buys tolerance of roughly five percent scattered loss. FEC shines when the RTT is very long or when the data is time-critical; it wastes capacity when the path is clean. Well-designed products tune the FEC ratio adaptively, dialing redundancy up on dirty paths and down on clean ones.

What Else Gets Rebuilt: Sessions, Encryption, Resume

Reliability and congestion control get the attention, but leaving TCP behind means rebuilding quieter machinery too, and evaluation questions hide in each piece.

• Session setup and authentication. UDP has no concept of a connection, so the protocol invents one: a handshake that identifies the two ends, authenticates the user (passwords, keys, or an existing SSH or TLS-style exchange), and agrees on parameters like the starting rate. Many products actually run this control conversation over a small TCP connection and use UDP only for the bulk data — worth knowing, because it means two kinds of firewall rules, not one.
• Encryption. There is no TLS socket to lean on, so the protocol must encrypt its own datagrams — reputable implementations negotiate keys with standard public-key methods and then encrypt every datagram with a modern symmetric cipher, the same construction TLS and SSH use. This is a place to verify rather than assume: ask which algorithms, and whether encryption is on by default. The full checklist lives in the security review article.
• Checkpointing and resume. Because the receiver already tracks which byte ranges have arrived, a well-built protocol can restart an interrupted multi-hundred-gigabyte transfer from the last verified block instead of from zero — often better resume behavior than plain TCP tools offer. If your transfers are huge, test this feature specifically.

Separating the Physics From the Marketing

Now you can evaluate a datasheet honestly. The table below pairs the common claims with what is actually true.

When an evaluation reaches the proof-of-concept stage, insist on a fair fight. Have the vendor's transfer race a tuned TCP baseline — correct buffer sizes, and parallel streams where appropriate — on your real path with your real files, not a lab demo against defaults. The honest gap that remains after tuning is the gap you would be paying for. Our companion articles give you both sides of that experiment: tuning TCP first and the measurement protocol.

Take this question list into the technical session — every item maps to a mechanism explained above, so the answers will tell you quickly how much substance sits behind the datasheet:

VENDOR EVALUATION QUESTIONS - accelerated transfer

1. What congestion control does the sender use, and how does it
   react to loss and to rising queueing delay?
2. Can we set a hard rate cap per flow? Who can change it?
3. Loss recovery: NACK-based retransmission, FEC, or both?
   Is the FEC ratio fixed or adaptive?
4. Which key exchange and which symmetric cipher protect the data
   datagrams? Is encryption on by default? Can it be disabled?
5. How is a transfer verified complete - per-block and whole-file?
6. What happens on interruption - resume from last verified block?
7. Which ports: control channel (TCP?) and data channel (UDP)?
   Can the data channel be pinned to ONE fixed port?
8. What does the benchmark baseline look like - tuned or default
   TCP, what RTT, what loss rate? May we re-run it on our path?

Where Acceleration Genuinely Earns Its Keep

Put the pieces together and the honest use-case list writes itself. UDP-based acceleration is the right tool when all three of these are true: the path is long (RTT roughly 80 ms and up), the volumes are large (tens of gigabytes and up, or hard deadlines on smaller sets), and measurement shows a wide gap between link capacity and what tuned TCP achieves — typically because loss and latency are compounding. Media companies moving footage between continents, research groups shipping instrument data, disaster-recovery replication across oceans: these workloads sit squarely in that triangle, and for them the technology is genuinely transformative.

Outside the triangle, the case evaporates quickly. On regional paths, tuned TCP fills the pipe. For many-small-files workloads, per-file overhead dominates and transport speed barely matters. For repetitive syncs, sending fewer bytes — compression and delta transfer, covered in our rsync and delta transfer series — beats sending bytes faster. And plenty of mid-size operations that measured first discovered their nightly window closed comfortably with tuned, scheduled TCP transfers; that is the honest reason our own products, like the Sysax FTP Automation scheduler, remain TCP-based tools: for the workloads most organizations actually run, well-managed TCP is enough, and the money stays in the budget.

The Version to Tell a Colleague

Accelerated transfer protocols are not magic UDP speed — UDP itself is just a blank envelope. They are TCP's guarantees rebuilt one layer up, with two policies swapped: a rate-based sender that pushes data at a steady target pace instead of one window per round trip, and surgical loss repair — NACK-driven resends and forward error correction — instead of halve-and-rebuild congestion backoff. On a long, slightly lossy pipe, those two swaps recover the enormous capacity TCP leaves idle; on a short clean pipe, there is nothing to recover. The claims are physics exactly as far as your latency and loss say they are, and marketing beyond that.

Next in the series: measure whether you actually have the problem, then apply the free fixes before anyone signs a purchase order. If a purchase does go ahead, the security review guide shows how to get the flow approved.

Frequently Asked Questions