Skip to content

feat(files): built-in file copy with reflink/CoW and .worktreeinclude support - #148

Merged
timvw merged 23 commits into
mainfrom
feat/builtin-file-copy
Aug 21, 2026
Merged

feat(files): built-in file copy with reflink/CoW and .worktreeinclude support#148
timvw merged 23 commits into
mainfrom
feat/builtin-file-copy

Conversation

@timvw

@timvw timvw commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Closes #144.

Built-in file copy: a [files] section materialises the untracked files a new worktree needs — .env, .envrc, local settings — with reflink/CoW where the filesystem offers it, plus .worktreeinclude support.

What it does

[files]
copy = [".env", ".envrc", ".claude/settings.local.json"]  # gitignore syntax
link = ["node_modules"]                                    # symlink instead
exclude = ["*.pem"]                                        # applied last, non-overridable
copy_ignored = false                                       # or: copy every ignored file
  • Runs automatically on wt create, wt checkout, wt pr and wt mr, after git worktree add and before the post_* hooks — including when the worktree already exists, so one made before [files] was configured catches up. Never fatal: a failure is reported and the worktree survives.
  • .worktreeinclude at the main-worktree root is unioned into copy, in gitignore syntax, for repos that would rather commit the list than a .wt.toml.
  • The list keys accumulate across layers (config file → repo .wt.toml.worktreeinclude), deduped first-seen; copy_ignored follows the ordinary scalar precedence chain, spelled wt.copyIgnored in git config.
  • wt copy [branch] does the same on demand, with --dry-run, --force and --from; --no-copy skips it once and WT_FILES_DISABLED=1 turns it off entirely.
  • Copies use clonefile(2) on APFS, FICLONE on Btrfs/XFS and copy_file_range(2) where available, through a bounded worker pool, falling back to a buffered copy.
  • wt info gains a Files section; wt config show reports copy_ignored; every command supports --format json.

Why it needs no wt trust

[files] cannot execute anything. It names paths inside two worktrees, and every path is checked against seven invariants, each with a test in cmd/files_security_test.go:

F1 source patterns resolve strictly inside the main worktree
F2 destination paths resolve strictly inside the new worktree
F3 .. segments are rejected at config-load time, escapes included
F4 symlinks are copied as symlinks, never dereferenced
F5 files git tracks are never copied — in either worktree, by construction
F6 an existing destination is never overwritten without --force
F7 nothing outside the two worktrees and os.TempDir() is read or written

Candidates come from git ls-files --others --ignored --exclude-standard, so a tracked file is never a candidate to begin with. VCS directories and registered worktrees of the same repo are excluded unconditionally.

Notable edge cases handled

  • A path the destination branch tracks is never written, --force included — including when it is absent there (deleted, or a sparse checkout), and including a tracked ancestor or subtree.
  • Case-insensitive destinations (APFS, NTFS, and also ext4 casefold or CIFS — probed, not assumed): two source paths differing only in case are reported as a collision rather than merged, directories included, before anything is created.
  • --dry-run predicts exactly what the real run does, down to which paths a copy claims before a link can have them.
  • No-clobber is TOCTOU-free (O_CREAT|O_EXCL); --force replaces atomically via a temp file and rename.

Testing

go test ./... -count=1, golangci-lint run (0 issues), GOOS=windows/GOOS=linux builds, and the declarative E2E suite (go run e2e/run.go -shells bash: 150 passed, 0 failed). Docs: docs/configuration.md and docs/examples.md.

Reviewed with 13 rounds of codex exec review; every finding was either fixed with a regression test that fails when the fix is reverted, or declined with a reason recorded in the loop.

timvw added 23 commits August 20, 2026 13:48
A fresh worktree has everything git tracks and nothing else, so the files
that make a checkout usable — .env, .envrc, .claude/settings.local.json —
are exactly the ones that are missing. Until now the answer was a
post_create hook running cp, which is shell-specific, silently wrong on
Windows, and needs `wt trust` before it runs.

Add a declarative [files] section instead:

    [files]
    copy = [".env", ".claude/settings.local.json"]
    link = ["node_modules"]
    exclude = ["*.pem"]
    copy_ignored = false

plus a committable .worktreeinclude at the main worktree root, sharing the
name with worktrunk, gtr and Claude Code's worktree feature so one file
works with all of them.

The three list keys accumulate across layers (config file → .wt.toml →
.worktreeinclude) rather than replacing each other: a user's "always copy
.env" has to survive a repo adding its own patterns, and a global "never
copy *.pem" has to hold against any repo. exclude is applied last and is
not overridable. copy_ignored is a scalar and follows the normal
precedence chain, including wt.copy_ignored in git config.

Materialisation runs on create/checkout/pr/mr, after `git worktree add`
and before the post_* hooks so a hook running `direnv allow` sees the file
that was just copied. Failure is non-fatal and never rolls back the
worktree. `wt copy` re-runs it on demand with --dry-run/--force/--from;
--no-copy and WT_FILES_DISABLED=1 suppress it.

Candidates come from `git ls-files --others --ignored --exclude-standard
--directory`, which resolves nested .gitignore files and core.excludesFile
for free and makes "tracked files are never copied" true by construction.
Files are cloned with FICLONE on Linux and clonefile(2) on darwin, falling
back to copy_file_range and then a buffered copy, so a multi-gigabyte
node_modules costs metadata rather than disk.

[files] deliberately does not require `wt trust`: unlike [hooks] it is
declarative data, held inside the two worktrees by seven invariants that
each have their own test in cmd/files_security_test.go.

New packages:
  internal/ignore  — hand-rolled gitignore matcher (git wildmatch with
                     WM_PATHNAME semantics), plus MayContain for walk
                     pruning
  internal/fileops — reflink/CoW with per-platform implementations and a
                     TOCTOU-free no-clobber via O_CREAT|O_EXCL

Also fixes plugins/wt/skills/wt/SKILL.md, which documented pattern
variables as {root}/{repo}/{branch} while the implementation uses the
dotted form — with missingkey=error that was a hard failure for anyone
following the skill.

Closes #144
…attern-trimming gaps

Codex review round 1:

- F2/F7: writing through a symlinked destination parent. withinRoot is
  lexical, so a destination worktree containing a tracked symlink
  (cache -> /tmp/outside) let cache/secret.txt land outside the worktree.
  noSymlinkComponents refuses to traverse a symlinked component, in
  copyOne, the directory pre-creation pass and linkConfiguredPaths.
- exclude was never applied to [files] link, so the documented
  "applied last and not overridable" was false for links.
- link named its paths literally and so bypassed the ignored/untracked
  candidate filter; it now checks the index directly (F5).
- accumulateFilePatterns trimmed patterns, turning the gitignore-valid
  "file\\ " into "file\\". Patterns are now passed through verbatim and
  ignore.LiteralPath resolves escapes where link needs a plain path.
…, dry-run never writes to source

Codex review round 2:

- A directory pattern only matched the directory itself. git reports
  candidates as leaf paths whenever a directory is not wholly ignored, so
  exclude = ["secrets/"] let secrets/key.pem through. ignore.Decide now
  evaluates every ancestor prefix before the path itself, as git does.
- A "!" in copy was ignored below a matched parent directory and under
  copy_ignored, because both short-circuited the matcher. Decide returns
  a tri-state so an explicit negation beats a blanket yes. This diverges
  from git, which cannot re-include below an excluded directory, and is
  documented: it only ever selects fewer files.
- The reflink probe created a temp file inside the *source* worktree,
  which "wt copy --dry-run" promises not to touch and which fails on a
  read-only source. It now compares filesystems first (no clone can span
  one) and probes inside the destination only.
…-run, stat link sources before excluding

Codex review round 3:

- F7: .worktreeinclude is committed, so a repo could ship it as a symlink
  to a file outside the worktree; os.Open followed it and wt info printed
  the resulting patterns back. It is now Lstat-ed and must be a regular
  file.
- wt copy --dry-run --force reported "skipped (exists)" for paths the
  real run would overwrite. dryRunResult takes force and mirrors both
  branches, including the refusal to replace a directory.
- link consulted the exclude list with isDir hard-coded to true, so a
  directory-only pattern such as "cache/" also excluded a link whose
  source is a regular file. The source is stat-ed first and the real
  directory bit is passed in.
- isTrackedPath uses a :(literal) pathspec so a link entry containing
  glob characters asks about that path rather than a pattern.
…nation directory

Codex review round 4:

- exclude accumulates with the repo's .wt.toml applied after the user's
  own config, so a committed exclude = ["!*.pem"] could undo a global
  "never copy *.pem" and then materialise it via copy_ignored. Negation
  is now rejected at config-resolution time in exclude and link, naming
  the pattern and its layer, and planner.excluded treats any decision as
  exclusion rather than only a positive one.
- F7: noSymlinkComponents deliberately stops before the leaf, which is
  right for a file but not for a directory — MkdirAll and Chmod both
  follow a symlink standing in for one, so a worktree with a tracked
  "cache -> /outside" had wt chmod a directory outside both worktrees.
  dirIsSafeToCreate checks the leaf as well.
Codex review round 5: dropCaseCollisions ran before the dry-run branch
and reached filesystemCaseInsensitive, which creates a temp file, so
"wt copy --dry-run" wrote into the destination worktree it promised not
to touch. caseInsensitiveWorktree flips the case of an entry that is
already there instead, and keeps the write probe only as a fallback for a
directory it cannot read.
The case-sensitivity probe still fell back to a write probe when the
destination could not be read or held nothing with a cased letter, so a
--dry-run could create a temp file there. Flip the case of the
destination's own name (and its ancestors') instead, which works on an
empty directory too.

A path named in both copy and link was reported as both copied and
linked by --dry-run, while the real run copies first and then skips the
link as existing. Feed the copy stage's planned paths into the link
stage so the preview matches.
--force removed the destination and then copied, so a copy that failed
afterwards (unreadable source, full disk) destroyed a file it only
promised to update. Copy to a temporary name beside it and rename over
it once the copy has succeeded.

git config variable names allow only alphanumerics and '-': 'git config
wt.copy_ignored true' fails with "invalid key", and a config file
holding that name does not parse at all. Read it as wt.copyIgnored, and
keep valueless boolean keys, which is how git spells true.
A relative --from was stored verbatim as a link entry's symlink target,
so the link resolved against the new worktree instead of the caller's
cwd. Make the resolved source absolute.

copyPlannedFiles returned before creating plan.dirs when no file was
selected, so a plan whose only content was an empty cache/ or
node_modules/ produced nothing.

--dry-run previewed a FIFO or device node reached through a selected
directory as a copy, while CopyFile refuses one. Predict the failure.
A "!" in copy lived in the same matcher as the positive patterns, so
last-match-wins let a later layer re-select it: a committed
.worktreeinclude naming cache/private.key undid the user's own
"!cache/private.key". Compile negations into a deny matcher applied
last, which is the same stance as rejecting "!" in exclude.

Directory creation errors were discarded, so a plan whose only content
was a directory could report "nothing to copy" having produced nothing.
Report each refusal and each mkdir failure as a failed entry.

Document the one thing --dry-run does write: the reflink probe inside
the destination worktree.
Copy candidates come from the source worktree's ignore rules, so a path
that main leaves untracked can be tracked on the branch being copied
into. --force was replacing that committed checkout with the untracked
copy, contradicting F5.

Read the destination index once per forced run and skip those paths,
in the preview as well as the real run.
A path the destination branch tracks but that is missing from that
worktree (deleted, or omitted by a sparse checkout) is still git's. The
previous guard only ran on an EEXIST collision under --force, so a plain
copy or a link happily materialised over it.

Read the destination index once per run and consult it before writing,
in both the copy and the link stage.
A branch can track "cache" as a plain file while the source ignores a
"cache/" directory. With that file absent from the destination worktree
the leaf check passed and EnsureParent/mkdir turned the tracked path into
an untracked directory. Walk the ancestors, in the directory stage too.
MkdirAllFrom chmod-ed unconditionally, so copying into a worktree whose
cache/ was deliberately 0700 silently widened it to the source's mode,
without --force and without a word in the output. Only a directory this
call creates takes the source's mode.
On APFS or NTFS a branch tracking "App.env" owns the file the source
calls "app.env", so an exact-match lookup let the guard through. Fold
the index behind a small trackedIndex type instead of threading the flag
through every call site.
git owns "vendor" as a directory when it indexes vendor/dep.js, even
where a sparse checkout or an rm has left the worktree without it, so a
link or a copy must not put a leaf there. Creating a directory at that
name is still fine — that is what checking the subtree out would do — so
only the leaf writers consult the new check.
wt checkout runs post_checkout on an existing worktree, so it owed the
same ordering as the create path: a hook expecting .env has to find it
whichever branch of the command it came through. This is also how a
worktree made before [files] was configured catches up.

Also reject a bare "!" pattern, which survived accumulation and was then
compiled away into a deny that protected nothing.
Copying cache/a.txt makes dst/cache exist, and a selected empty cache/ is
created outright — either way the real run's link stage finds the name
taken. The preview only knew about copied leaves, so it promised a link
the run would skip.
A destination that folds case sees one path where a case-sensitive source
holds two. The collision pass only saw files, and ran after the selected
directories had already been created — so a source with both Cache/ and
cache/ merged the two trees silently, and an empty Cache/Config/ created
up front turned a later cache/config into a copy onto a directory that
the dry run had predicted would succeed.

Directories now go through the same pass, before anything is created, and
a path whose parent already exists under another spelling is refused with
it. The reason stays case-insensitive collision with <path>.
F3 says a ".." segment is refused when the config is resolved, but the
check read the raw pattern. A link entry spelled '\.\./secret' passes it
and only becomes ../secret once LiteralPath resolves the escapes — caught
by withinRoot on the way out, which is a guard, not the promise. Validate
the de-escaped body as well.
…ries

Two gaps the review found.

The case-fold probe was gated on darwin/windows, so an ext4 casefold
directory or a CIFS mount on Linux went through as case-sensitive and
collisions there were resolved by whoever wrote last. The probe itself is
platform-independent — it flips the case of a name that already exists and
compares inodes — so run it everywhere and keep the platform only as the
fallback for a worktree with no cased name to test.

A plan whose whole content was an empty selected directory changed the
worktree while both runs printed "Nothing to copy". Directories that have
to be brought into existence are now reported as created, by the dry run
and the real one alike; one that is already there still says nothing.
wt checkout learned this; the other three commands that put you in a
worktree did not, so a worktree made before [files] was configured caught
up on checkout but not on create, pr or mr. The [files] docs promise all
four. Same no-op cost on a worktree that already has everything.
@timvw
timvw enabled auto-merge (squash) August 21, 2026 05:56
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.58824% with 335 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.76%. Comparing base (7947a5f) to head (beac024).

Files with missing lines Patch % Lines
cmd/files.go 80.93% 110 Missing ⚠️
cmd/copy.go 9.72% 65 Missing ⚠️
cmd/info.go 0.00% 56 Missing ⚠️
internal/ignore/wildmatch.go 75.70% 26 Missing ⚠️
internal/fileops/fileops.go 66.00% 17 Missing ⚠️
cmd/checkout.go 0.00% 12 Missing ⚠️
cmd/create.go 0.00% 12 Missing ⚠️
cmd/pr.go 0.00% 12 Missing ⚠️
internal/fileops/reflink_linux.go 80.55% 7 Missing ⚠️
internal/fileops/samefs_unix.go 0.00% 6 Missing ⚠️
... and 3 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #148      +/-   ##
==========================================
+ Coverage   44.34%   50.76%   +6.42%     
==========================================
  Files          35       43       +8     
  Lines        3689     4810    +1121     
==========================================
+ Hits         1636     2442     +806     
- Misses       2053     2368     +315     
Files with missing lines Coverage Δ
cmd/config_cmd.go 74.60% <100.00%> (+0.83%) ⬆️
cmd/examples.go 69.64% <ø> (ø)
cmd/root.go 78.40% <100.00%> (+1.57%) ⬆️
cmd/config.go 90.67% <97.72%> (+1.44%) ⬆️
internal/fileops/copy_range_linux.go 68.75% <68.75%> (ø)
internal/fileops/samefs_unix.go 0.00% <0.00%> (ø)
internal/ignore/ignore.go 95.41% <95.41%> (ø)
internal/fileops/reflink_linux.go 80.55% <80.55%> (ø)
cmd/checkout.go 0.00% <0.00%> (ø)
cmd/create.go 0.00% <0.00%> (ø)
... and 6 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@timvw
timvw merged commit 11bdcc9 into main Aug 21, 2026
16 checks passed
@timvw
timvw deleted the feat/builtin-file-copy branch August 21, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Built-in file copy: reflink/CoW + .worktreeinclude support

1 participant