Designing a File Interface Contract

"Where is the spec for the eligibility feed?" "It's in the export job. And in the import job. And in the head of whoever set it up." The most dangerous file interface in your estate is the one that works. It has run for years. The producer's export job and the consumer's import job mirror each other perfectly, and nothing enforces that mirror except habit. The format lives in one team's code, and the schedule in another team's scheduler. The meaning of a missing file lives in the memory of someone who left. Then a field gets widened, or the export runs an hour late for the first time ever. The two teams discover they had different interfaces in mind all along. The spec, it turns out, was an email from three reorganizations ago.

The fix is a document: the file interface contract. It is a written specification of everything both sides must agree on for the interface to work. It covers the file, its movement, its timing, its acknowledgment, and its behavior when things go wrong. It is not glamorous, and it is the difference between an interface and a coincidence. This article walks through the contract element by element and ends with a complete worked example you can copy and adapt. It is part of our Files as Integration Glue series. The opening article made the case for file interfaces, and this one is where doing them well begins.

Why the Agreement Must Be Written

A file interface is implemented twice: once in the producer's export logic, once in the consumer's import logic. Those two implementations must agree exactly — on delimiters, on field order, on what an empty batch looks like, on what time "nightly" means. Yet no compiler, no schema check, and no handshake verifies that they do. In an API integration, a typed interface catches many mismatches at build time. In a file integration, the only thing standing between agreement and quiet divergence is a shared description that both teams treat as authoritative.

That is all a contract is. Not a legal document (though for an external partner it may be attached to one), not bureaucracy — an engineering artifact, like a network diagram or a runbook. The test of a good one is simple: a new administrator on either side, holding only the contract, could operate, troubleshoot, and correctly modify their half of the interface. If knowledge beyond the document is required, the document is not finished.

Write one for any interface that crosses a team boundary, and consider it non-negotiable for any interface that crosses a company boundary. The half hour it takes to write down what you believe the interface does has a second benefit people underestimate. The two sides discover, in a meeting room instead of an incident call, that they believed different things.

Meridian Parts wrote one down for the first time when the supplier price feed changed hands. The producer's team believed the file was a full snapshot. The consumer's loader had been treating it as a delta and merging. That explained a small population of discontinued parts that had refused to disappear for two years. Neither side had been wrong about its own half. Both had been wrong about the other's, and the contract meeting was the first time anyone had said either belief out loud. The fix was one line in section 2 and a one-time purge, and the discontinued parts finally left.

The Anatomy of the Contract, in One Picture

The diagram below shows a typical file interface. A producer pushes a file to an exchange server, a consumer collects it, and an acknowledgment comes back. Each contract element is attached to the part of the picture it governs. Everything labeled here becomes a numbered section in the document.

The exchange server in the middle is deliberate. Direct system-to-system delivery is possible, but a neutral handoff point decouples the two sides further. It gives both of them one place to look when something goes wrong. This pattern is explored properly in our server-to-server exchange patterns series. The contract names this middle ground explicitly: which server, which folders, who may read and write each one. A neutral folder has no side to take, which is its entire charm.

Name the Parties, Then the Data

The contract opens by identifying things people will otherwise assume. First the interface itself: give it a short interface ID ("IF-042") and a human name ("employee eligibility feed"). The ID ends ambiguity in tickets and log messages. Three interfaces between the same two systems is common. "The HR file" stops being a useful phrase the day the second one appears.

Then identify the parties: which team owns the producer side and which owns the consumer side. Include — critically — the contact path for each, written as roles or shared mailboxes rather than individual names, because individuals leave. The pattern is spelled out in flow ownership and contacts. Include the escalation route for out-of-hours failures if the interface matters at 3 a.m. Individuals leave; shared mailboxes merely fill up.

Then the data, and here the contract makes its first real design decision. Is each file a snapshot — the complete current state, every record, every time? Or is it a delta, containing only what changed since the last file? Snapshots are larger but forgiving: miss one, and the next file heals everything. Deltas are compact but stateful: miss one and the consumer's picture is silently wrong until someone notices. The contract states which model applies, and for deltas, how a missed file is detected and recovered. It also records expected volume ("about 8,200 records; investigate outside 5,000–12,000"), which later becomes a cheap sanity check on both sides.

Specify the File Down to the Byte

This is the longest section of most contracts, and the place where vagueness costs the most. It has three layers.

The name. Spell out the exact pattern with every token defined: EMPL_ELIG_YYYYMMDD_NN.csv. The date token is the business date the data describes (not the production timestamp — those differ at midnight). And NN is a two-digit sequence for resends within one day. Names are load-bearing in file integration: scripts parse them, monitors match on them, humans sort by them. Our guides to naming convention design and datestamp formats that sort cover the reasoning; the contract records the outcome.

The format. Specify the delimiter, quoting rules, character encoding, line endings, and whether a column-header row is present. Specify header and trailer records and control totals. Include a field-by-field table: name, position, type, maximum length, required or optional, an example value. Every one of those words hides a failure mode — a comma inside a company name, an accented character in a legacy code page. The whole subject gets its own article in this series, choosing flat-file formats that age well. The contract either contains the field table or points to the format spec that does. It includes a version number so both sides can say which edition they implement. "The latest one" is not a version number.

The empty batch. Decide what happens on a day with no data. The producer sends a file anyway — header and trailer present, zero detail records — rather than sending nothing. This rule looks pedantic and is anything but. If "no file" is a legitimate outcome, then a broken producer is indistinguishable from a quiet day, and your monitoring is blind. If a file always arrives, absence always means failure.

Schedule, Delivery, and the Completion Signal

The schedule clause answers four questions. When is the file produced? By when must it be delivered? In which time zone are those clocks? Name it explicitly, because "2 a.m." means different things in different rooms, and daylight-saving shifts move the goalposts twice a year. And on which calendar — every day, business days only, and if so, whose holidays? A cutoff without a time zone and a calendar is a guess. The contract also defines "late": the moment at which the consumer stops waiting and starts the error procedure. That is usually well before the moment anyone panics. Panic keeps its own schedule, and it is not in the contract.

The delivery clause records the mechanics: protocol (SFTP, FTPS), direction, and who initiates. Does the producer push to the exchange point, or does the consumer pull from the producer? Push and pull distribute credentials, firewall openings, and failure visibility differently; the tradeoffs live in our server-to-server patterns series. Then the concrete details: host, folder paths for inbound files, acknowledgments, and rejects, the account used, and how its credentials are rotated.

Finally, the completion signal — how the consumer knows the file is entirely written, not still streaming in. There are two standard answers. One is upload-then-rename: the producer writes name.tmp and renames it to the final name only when done. So the final name only ever appears complete. The other is a marker file (a tiny second file, say .done, written after the data file). One of them must be in the contract; a consumer that reads files with no completion rule will eventually import half a batch. The mechanics of both are covered in marker and control files.

On the implementation side, each half of this clause maps to ordinary automation. The producer's push is a scheduled job, and the consumer's pickup is either a scheduled poll or a folder watch. A tool like Sysax FTP Automation covers both shapes. It provides scheduled tasks for the cutoff-driven push and folder monitoring so the consumer's side reacts the moment the file (or its marker) lands. It provides email notifications when a transfer fails. So the contract's timing clauses become configuration rather than custom code.

Acknowledgment and Error Behavior

A file interface is silent by default: the producer learns nothing from a successful delivery except the absence of complaints. The contract fixes this with an acknowledgment clause. It specifies what the consumer sends back (typically a small ack file whose name mirrors the data file). It specifies what the ack contains (status, records read, whether the trailer's control totals matched) and where it is written. And it specifies the part most drafts forget: the deadline, plus what the producer does when the deadline passes with no ack. The full menu of acknowledgment designs, from nothing at all to signed receipts, is the subject of its own article in this series. The contract records which pattern this interface uses. The absence of complaints also describes a consumer that has not looked yet.

The error clause is the contract's other half of honesty. It answers, in advance, the questions that otherwise get answered by escalation. What does the consumer do with a file that fails structural checks — reject the whole thing, or process good records and reject bad ones? Where do rejects go, in what format? How does the producer resend — a full replacement file, under what name, and how does the consumer avoid double-processing the records it already applied? Each answer references a mechanism covered in error handling across a file interface. But the decisions belong here, in writing, agreed before the first bad file rather than during it.

Two clauses round out the operational picture. A verification clause, if the data warrants it: a checksum accompanying each file, so corruption in transit is detectable — see checksum files and manifests. And a records clause: both sides log deliveries, and the exchange server's own activity log is the neutral record when their accounts differ. If the exchange point is a Windows server running Sysax Multi Server, that activity log shows every upload and download with account and timestamp. In that case, the log is written to file and to a database. That is exactly the evidence two disagreeing teams need.

A Complete Worked Contract

Here is a full contract for a fictional nightly feed, compressed to the length real ones usually reach. Copy it, replace the specifics, and delete nothing without deciding the question it answers some other way.

FILE INTERFACE CONTRACT  IF-042  "EMPL-ELIG"          format version 03

1. PARTIES
   Producer:  HR platform team            hr-platform@example.com
   Consumer:  Benefits operations team    ben-ops@examplepartner.com
   Escalation: each side's on-call rotation; phone bridge for P1.

2. DATA
   Employee benefits eligibility. FULL SNAPSHOT each run (not delta).
   Typical volume 8,200 detail records; investigate outside 5,000-12,000.

3. FILE
   Name:     EMPL_ELIG_YYYYMMDD_NN.csv
             YYYYMMDD = business date; NN = 01, incremented on resend.
   Format:   CSV, UTF-8 (no BOM), CRLF line endings, comma delimiter,
             fields containing comma/quote/newline wrapped in double
             quotes, embedded quotes doubled. Field table: Appendix A.
   Records:  H,EMPL_ELIG,YYYYMMDD,03        (header: id, date, version)
             D,<empid>,<fields...>          (one per employee)
             T,<detail count>,<sum of premium_amount>   (trailer)
   Empty batch: file still sent, zero D records, T count 0.

4. SCHEDULE
   Produced daily including holidays. Uploaded complete by 02:00
   America/Chicago. LATE at 02:30: consumer opens incident, both
   sides notified. Missing entirely at 06:00: escalate P1.

5. DELIVERY
   SFTP push by producer to exchange.example.com as account emplfeed
   into /feeds/empl_elig/in/. Upload as .tmp, rename to final name
   when complete (rename is the completion signal).
   Key-based authentication; keys rotated per the partner standard.

6. ACKNOWLEDGMENT
   Consumer writes EMPL_ELIG_YYYYMMDD_NN.ack to /feeds/empl_elig/ack/
   within 60 minutes of pickup. Contents: STATUS=OK|REJECTED,
   RECORDS_READ, TRAILER_MATCH=Y|N. No ack by 04:00: producer treats
   the delivery as failed and phones the consumer on-call.

7. ERRORS
   Structural failure or trailer mismatch: whole file rejected,
   STATUS=REJECTED ack plus .rej detail file in /feeds/empl_elig/rej/.
   Producer corrects and resends FULL replacement as NN+1. Consumer
   processes highest NN per business date only (earlier NN discarded).
   Record-level policy: none - this feed is all-or-nothing.

8. CHANGES
   Minimum 30 days notice via change request to both mailboxes.
   Format changes run parallel (old + new) for two cycles minimum.
   This document is versioned; change log below.

CHANGE LOG
   v03  Mar 14  added TERM_DATE field (position 9), parallel-run done
   v02  Nov 02  trailer gains premium_amount hash total
   v01  Jun 20  initial agreement

Two details deserve a second look. The resend rule in section 7 — "highest NN per business date wins" — quietly solves the duplicate problem. The consumer has one deterministic rule for which file is authoritative, however many arrive. And section 4 distinguishes late from missing: different clocks, different responses. Contracts that only define the happy path get renegotiated during incidents, which is the worst possible venue.

Negotiating It, and Keeping It Alive

Expect a predictable tension. Producers want freedom: loose windows, the right to add fields, minimal ceremony. Consumers want rigidity: exact formats, tight deadlines, guarantees. Neither is wrong — the contract is where the tension gets settled once instead of nightly. A fair default: the side that wants a change absorbs its cost, and the side with the regulatory or downstream exposure sets the strictness floor. Write the first draft yourself regardless of which side you are on; the author frames the defaults. I have never regretted holding the pen, and I have regretted reviewing.

Sometimes the other side will not engage — a vendor with no interest, an internal team with no time. Write the document anyway, as an interface memo. It says "here is what we observe your system doing, and what ours does; we will operate to this and treat deviations as incidents unless you correct it." A one-sided record beats none, and it converts every future surprise into a documented deviation rather than a mystery. The vendor may never read it; your successor will.

Finally, keep it alive. The contract lives somewhere both sides can read — not in one team's private wiki. It carries its own version number and change log, exactly like the worked example. Reread it after every incident the interface causes (the incident usually found a gap). Reread it whenever either team's ownership changes hands, and before any planned change. The article on keeping documentation current covers the habit in general. Changing the interface itself without breaking the other side is a craft of its own, covered in evolving a file interface. It works precisely because the contract gives you a baseline to evolve from.

From Agreement to Operation

A file interface contract is eight decisions written down: who, what data, what file, what schedule, what delivery, what acknowledgment, what error behavior, what change process. None of the decisions is hard. All of them are expensive to make implicitly, because implicit decisions get made twice — once on each side, differently. The worked document above is deliberately short; an afternoon and one meeting produce it. It pays for itself the first time a file is late or a field is disputed. It also survives reorganizations, which the email did not.

From here, go deeper on the two clauses with the most engineering inside them. Read choosing flat-file formats that age well for section 3 of your contract, and acknowledgment patterns for section 6.

Frequently Asked Questions