Bulk Move Pitfalls: Long Paths, ACLs, and Open Files

Every migration meets the same six enemies. The reason is not that administrators keep making new mistakes. A bulk move touches every file ever created on a server. Somewhere in years of accumulation there is always a path four characters short of the limit. There is a permission entry pointing at an account that left the company and a folder tree that quietly rewrites its own timestamps. And there is a file that has been open in an application since anyone can remember. Daily transfer jobs never meet these files. Your migration will meet all of them.

This article is the field guide to six classic wreckers. They include long paths, permissions that translate wrong, timestamp drift, hard links and their special-file cousins, and open files. The sixth is the seed pass that flattens the network at ten in the morning. Each comes with the symptom you will actually observe and a mitigation that works. Read it before the seed copy, when every fix is cheap; each pitfall costs ten minutes in planning and an evening in a log file.

This is the closing article of our Bulk Internal Moves with Robocopy and Rsync series. It leans on the engine details from the robocopy and rsync deep-dives.

The Pitfall Map

The one-table version, for the wall above your desk. Details, evidence, and commands follow.

Long Paths: The Limit Nobody Remembers Until It Bites

Windows historically caps a full path — drive letter or share prefix, every folder name, backslashes, file name — at around 260 characters, the MAX_PATH limit. Deep trees flirt with it constantly: nested project folders, verbose names, files saved by people who write sentences into file names. Robocopy itself copes — it uses extended-length path handling internally and copies beyond the limit without complaint. And that is precisely the trap: the copy succeeds, and the problem surfaces afterward. Explorer or an application with ordinary path handling cannot open, save, or delete the deepest files at their new home.

Migrations make paths longer in a stroke of bad luck. Suppose the new root is deeper than the old one — \\newserver\departments\engineering\ replacing \\oldserver\eng\. In that case, every path in the tree grows by the difference, and files that lived comfortably near the limit cross it silently. The mitigations are all pre-move. Keep the destination root at most as long as the source root. Hunt near-limit paths during inventory (the command is in the checklist below) and get the owners to shorten the worst offenders. A rename today beats a support ticket forever. And if the tree itself is the problem, the migration is the moment to restructure it deliberately. The companion for that decision is designing directory trees.

ACLs That Translate Wrong

Permissions are the pitfall with the widest blast radius, because they fail in both directions. Users can be locked out of their own work, or a sensitive folder can be readable by everyone. Which failure you get depends on what the move crosses.

Within one domain, Windows to Windows, ACLs travel intact when you ask (/COPYALL). The entries reference domain accounts by SID (the security identifier behind every user and group). Those SIDs mean the same thing on both servers. What travels less gracefully is history: entries for deleted accounts arrive as unresolvable "unknown account" SIDs. They are mostly harmless clutter, but a migration is a fine excuse to tidy them, and a permissions review beforehand shortens the tidy.

Across domains or into a workgroup, the same SIDs mean nothing at the destination. Copying ACLs faithfully copies nonsense: the entries land, resolve to nobody, and the effective result is folders governed by leftover inheritance and luck. The honest plan is a rebuild — copy the data, then apply a designed permission structure on the new server. Grant through groups rather than dragging ten years of individual grants along.

Across platforms — NTFS to a Linux/NAS filesystem or back — the permission models themselves differ. Inheritance rules, deny semantics, and granularity do not map one-to-one, and whatever automatic translation occurs will surprise someone. Treat it like the cross-domain case: rebuild deliberately, and read permission models explained for why the models cannot simply be transliterated.

Ownership rides along with this pitfall and is easy to forget. It is a separate slice of metadata (robocopy's O in /COPY:DATSOU, rsync's -o as root). When it does not travel, every file arrives owned by the migration account. Quotas count against the wrong person, "created by" trails go blank, and any process that keys on ownership misbehaves quietly. Copy it where the accounts translate; where they do not, assigning ownership is part of the rebuild.

Whatever the path, the mitigation ritual is identical. During the delta week, test with real accounts — one user from each major group opening real folders on the new server. And spot-check with icacls exports on both sides. Monday morning is the wrong place to discover which of the two failure directions you built.

Timestamp Drift

Timestamps look cosmetic until you remember what reads them: your own delta passes ("copy what is newer"), users sorting by modified date, retention scripts deleting by age. A migration that mangles times breaks all three.

Drift has three classic causes. Granularity: some filesystems store times coarsely — the FAT family rounds to two-second boundaries. So a file copied to certain NAS or USB targets gets a time that no longer exactly matches the source. Every later pass sees "different timestamp," concludes "changed," and recopies the entire share; the giveaway is a second delta as large as the seed. Robocopy's /FFT flag fixes the comparison by tolerating two-second differences. Daylight-saving offsets: a whole tree suddenly differing by exactly one hour is the DST signature. It is born of how different filesystems record local versus universal time; robocopy's /DST compensates. Tools that stamp copy time: copy without time preservation and every file or folder claims it was born on migration night. Rsync's -a preserves file times by default. Robocopy needs /DCOPY:T for directory timestamps specifically, an omission you notice only after cutover.

Hard Links, Junctions, and Other Special Files

Most files are one name pointing at one body of data. The exceptions wreck naive copies.

Hard links are multiple directory entries sharing a single file body. Copy them without link awareness and each name becomes an independent full copy. The target grows mysteriously larger than the source, and any application relying on the link relationship (deduplicated backup trees, build outputs) breaks. Rsync preserves them with -H at a memory-and-time cost worth paying only where links actually exist. Robocopy has no hard-link preservation and will duplicate the content for each name. If a Windows tree depends on hard links heavily, plan around that honestly. In that case, move it with an image-level or archive-based method instead of a file copy. Inventory tells you whether you care: on Linux, find -links +1 lists linked files in seconds.

Junctions and symbolic links are names pointing at other paths. There are two dangers. A link pointing back up its own tree can send a recursive copy into a loop (robocopy's /XJ excludes junction points; document the exclusion). And a link with an absolute target — \\oldserver\share\tools or /srv/olddata/bin — copies "successfully" and then dangles the day the old server dies. Audit absolute link targets during inventory and re-point them as part of cutover.

Sparse files are large files that are mostly unwritten space, like some VM disks and database files. They can balloon to their full nominal size when copied by tools that do not preserve sparseness. If multi-hundred-gigabyte sparse files live on the share, test one before the seed and budget target space for the worst case.

Open Files

The signature is unmistakable: the same handful of files failing with sharing violations in every pass log. Examples are the mailbox archive open all day, the accounting package's data file, and the database nobody admits is running on a file share. No copy flag defeats an exclusive lock; robocopy's backup mode (/B) reads past permissions, not locks, a distinction the robocopy article draws sharply.

The mitigation is a sequence, not a trick. First, keep retries low (/R:2 /W:5) so locked files cost seconds, not stalled nights. Second, let the delta rhythm work: most "always open" files are actually closed at 3 a.m., and a nightly pass catches them eventually. Third, the write freeze is the real answer. On cutover night, sessions are disconnected and the services that hold files are stopped, so the final pass meets closed files. The freeze checklist must therefore include the lock holders: the archiving service, the application services, the workstation someone left running a spreadsheet since Tuesday.

Finding the lock holder is half the fix. On a Windows file server, the shared-folder management console's open-files view shows which account holds which file open over the network. Locks held by local services show up in the services list once you know to suspect them. On Linux, lsof +D /path answers directly. Do this reconnaissance during the delta week — the same names appear night after night, and each one becomes a line on the freeze checklist.

Two special cases deserve their own plan. Live databases should not be file-copied at all, even frozen. Export or back them up with the database's own tooling and restore on the new side. A file-level copy of a database in use is corruption with a completion percentage. And suppose a file genuinely can never close — a system that cannot be stopped even during the freeze. In that case, the platform's snapshot facility (a volume shadow copy on Windows) can provide a frozen image to copy from. That comes at the cost of extra tooling to mount it. Treat that as the advanced path, needed far less often than feared once a real freeze exists.

Saturating the Network During Business Hours

The seed pass is a bandwidth monster by design, and the complaint it generates is never "the migration is slow." It is "everything is slow": calls stuttering, applications timing out, a helpdesk fielding vague misery with no server showing a fault. Migration traffic on a shared link is a denial of service you scheduled for yourself. It is worst on WAN links between sites, where the copy competes with every business application at once.

Mitigate in layers. Schedule first: run heavy passes overnight and on weekends. The discipline of windows, service accounts, and jobs that actually start is covered in our scheduled jobs series. Throttle what must run by day: rsync has --bwlimit, a clean rate cap. Robocopy has /IPG:n, which inserts a gap of n milliseconds between 64-kilobyte blocks. That is crude, effective, and incompatible with /MT multithreading, so a throttled daytime pass is also a single-threaded one. Respect the other schedules: the nightly backup window is the classic collision — a seed and a backup fighting for the same disks and links helps neither. And on WAN legs, the tuning lore in rsync over WAN applies directly. If what remains after the migration is a recurring inter-site feed, retire the hand-throttled migration scripts and give the job to a proper scheduler. Sysax FTP Automation runs mirror and synchronization tasks over SFTP, FTPS, or FTP on defined schedules. It provides retry handling and email notification when a run fails, which is the polite way to use a shared link forever. And if that recurring leg crosses an untrusted network, put a secure transfer server at the receiving end rather than exposing a file share to the world. An example is Sysax Multi Server (SFTP, FTPS, HTTPS, with activity logging).

The Pre-Move Pitfall Hunt

Every pitfall above is findable before the seed. Fold this into the inventory phase described in the series opener on planning the big copy:

# Windows: paths within striking distance of the limit
Get-ChildItem \\oldserver\eng -Recurse |
  Where-Object { $_.FullName.Length -gt 220 } |
  Select-Object -ExpandProperty FullName

# Windows: largest files (sparse/resume risks) - top 20
Get-ChildItem \\oldserver\eng -Recurse -File |
  Sort-Object Length -Descending |
  Select-Object -First 20 FullName, Length

# Linux: hard-linked files (link count > 1)
find /srv/data -type f -links +1 -printf '%n %p\n' | sort -rn | head

# Linux: symlinks with absolute targets (future dangling links)
find /srv/data -type l -lname '/*' -printf '%p -> %l\n'

# Linux: what is open right now under the tree
lsof +D /srv/data | head -40

Then rehearse: run your full hardened command against a test tree that contains the skeletons. Include a 250-character path, a locked file, a hard-linked pair, a deny ACL — and read the log. A pitfall met in rehearsal is a footnote; the same pitfall met at 2 a.m. on cutover night is the story everyone tells about the migration for years.

Forewarned Is Migrated

There are six enemies and six fixes, all cheap before the seed and expensive after cutover. Keep the destination root short and hunt long paths. Decide whether permissions translate or get rebuilt, and test with real accounts. Preserve timestamps and add the granularity flags your targets need. Inventory hard links, junctions, and sparse files before they inventory you. Let low retries, nightly rhythm, and the freeze absorb open files. And keep the big passes off the business-hours network. The verification pass will tell you how well you did — every pitfall here reappears there as a residue class. That is why verifying nothing was left behind is the natural next read. Alongside it belongs the cutover runbook, where the last of these battles is won.

Frequently Asked Questions