Isolate each parallel qmd render in a throwaway project copy

[VincentGuyader]

Problem

compile_qmd_course() renders qmds with several future::multisession workers inside the same quarto project. Quarto does not support concurrent renders sharing one project (quarto-dev/quarto-cli#2749). Under load this produces random Failed to render errors with two signatures, both reproduced and captured on ThinkR's unicorn CI with debug = TRUE:

1. ERROR: Unable to read the extension '<name>' : transient failure while resolving the extension format (a few seconds window, several workers hit at once);
2. SyntaxError: Unexpected non-whitespace character after JSON in crossrefIndexForOutputFile : the shared .quarto/ crossref index corrupted by interleaved writes (up to 287 occurrences in a single CI job). This corruption persists on disk, so retries cannot recover.

The failures are load dependent (slower renders widen the race window), which made them look random: a job failed, got retried on a quieter machine, and passed.

Fix

Each render now runs inside a throwaway copy of the quarto project (build_isolated_project()):

• the repository tree is exposed through symlinks, so here::here(), data paths and cross-module references keep resolving;
• _quarto*.yml, _extensions/ and the chapter of the target qmd are real private copies;
• each render gets its own fresh .quarto/ state;
• outputs (html, extracted media) are copied back to the real chapter afterwards (collect_isolated_outputs()).

Controlled by the new isolate argument of render_single_qmd() (default getOption("squash.isolate_render", TRUE)); on isolation failure it falls back to rendering in place. render_single_qmd() also now reports conditionMessage() of the underlying error instead of discarding it.

Validation

• Without the fix: red on the very first run of the unicorn niveau1 harness in the CI image; 4/9 red jobs on a stressed CI pipeline.
• With the fix: 6/6 consecutive local harness iterations green and 22/22 green CI jobs over two full unicorn pipelines (40371, 40372 on the forge), zero retries. Per-render overhead of the isolation is negligible (symlinks + copy of one chapter and 672K of extensions).
• New unit tests cover the isolated-project layout (private copies vs symlinks, .here anchor, no shared .quarto) and the copy-back of outputs.

Note: regenerating the docs migrated RoxygenNote: 7.3.2 to Config/roxygen2/version: 8.0.0 (roxygen2 8.x behavior).

Full investigation write-up in forge MR thinkr/thinkrverse/unicorn!1252.

[VincentGuyader]

Full test suite run in the CI image (bk_santacruz, R 4.6.1, quarto 1.10.18, Chrome 151):

TOTAL: pass=58 fail=0 warn=0 skip=1
test-add_extension.R            4  0
test-check_quarto_version.R     1  0
test-clean_rendering_files.R    1  0
test-compile_qmd_course.R      22  0
test-copy_if_not_already_exist  3  0
test-create_template_html.R     3  0
test-extract_html_slides.R      3  0
test-fetch_future_settings.R    2  0
test-isolate_qmd_render.R      14  0   (new)
test-render_single_qmd.R        5  0

Also visible in the run: for qmds without a quarto project above them (several existing test fixtures), isolation falls back to rendering in place as designed ("Could not isolate ..., rendering in place").

[codecov-commenter]

Codecov Report

❌ Patch coverage is 93.58974% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.94%. Comparing base (05be0a0) to head (8898afd).

Files with missing lines	Patch %	Lines
R/isolate_qmd_render.R	94.35%	11 Missing ⚠️
R/render_single_qmd.R	89.74%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #23      +/-   ##
==========================================
- Coverage   98.97%   96.94%   -2.03%     
==========================================
  Files          12       13       +1     
  Lines         390      622     +232     
==========================================
+ Hits          386      603     +217     
- Misses          4       19      +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR addresses non-deterministic failures when rendering multiple Quarto .qmd files in parallel by ensuring each render runs in an isolated, throwaway copy of the Quarto project, avoiding concurrent access to shared .quarto/ state and extension resolution.

Changes:

• Add isolated-project helpers (build_isolated_project(), collect_isolated_outputs()) and enable isolation by default via render_single_qmd(isolate = getOption("squash.isolate_render", TRUE)).
• Improve render failure reporting by including the underlying conditionMessage() in render_single_qmd() messages.
• Add unit tests for isolated-project layout and copy-back behavior; update NEWS and metadata (dev version + roxygen config).

Reviewed changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
R/isolate_qmd_render.R	Introduces isolated-project creation and output collection logic to avoid shared Quarto state during parallel renders.
R/render_single_qmd.R	Adds isolate option and integrates isolated rendering + cleanup; improves error messaging.
tests/testthat/test-isolate_qmd_render.R	Adds tests validating isolation layout (symlinks vs copies) and output copy-back behavior.
NEWS.md	Documents the new isolated rendering behavior and improved error reporting.
DESCRIPTION	Bumps dev version and updates roxygen configuration metadata.
man/render_single_qmd.Rd	Documents the new isolate argument.
man/squash-package.Rd	Updates package author listing.

Files not reviewed (2)

• man/render_single_qmd.Rd: Generated file
• man/squash-package.Rd: Generated file

Suppressed comments (7)

R/isolate_qmd_render.R:57

• list.files() is called with no.. = TRUE, which is not available on older supported R versions (and isn’t needed here). This can break CI on older R releases.

  for (entry in list.files(proj, all.files = TRUE, no.. = TRUE)) {

R/isolate_qmd_render.R:83

• list.files() is called with no.. = TRUE, which is not available on older supported R versions (and isn’t needed here). This risks failures on older R / Windows.

      for (entry in list.files(real_cur, all.files = TRUE, no.. = TRUE)) {

R/isolate_qmd_render.R:98

• list.files() is called with no.. = TRUE, which is not available on older supported R versions (and isn’t needed here). This will error on older R versions.

        from = list.files(
          real_cur,
          full.names = TRUE,
          all.files = TRUE,
          no.. = TRUE
        ),

R/isolate_qmd_render.R:72

• file.symlink() failure is not checked here. On platforms where symlinks are disallowed, this silently produces an incomplete isolated project and makes renders fail in surprising ways.

      file.symlink(
        from = file.path(proj, entry),
        to = file.path(iso_cur, entry)
      )
    }

R/isolate_qmd_render.R:89

• file.symlink() failure is not checked here. If creating symlinked siblings fails (notably on Windows), the isolated project can miss required files and the render will fail without a clear cause.

          file.symlink(
            from = file.path(real_cur, entry),
            to = file.path(iso_next, entry)
          )
        }

tests/testthat/test-isolate_qmd_render.R:2

• Call the symlink capability guard at the start of the test so environments without symlink support skip cleanly instead of failing part-way through.

  # fake repo: root/(data.csv)/courses/(_quarto.yml, _extensions, M01/S01/file.qmd)

tests/testthat/test-isolate_qmd_render.R:50

• Call the symlink capability guard at the start of the test so environments without symlink support skip cleanly instead of failing unpredictably.

  root <- tempfile(pattern = "iso_repo")

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[VincentGuyader]

False positive: the value fed to strsplit() derives from dirname(qmd), and on Windows dirname() always returns /-separated paths (documented in ?basename, section Behaviour on Windows; implemented via R_UTF8fixslash in src/main/util.c). The normalizePath() result with backslashes never reaches the split. The actual Windows breakage was the unchecked file.symlink() (see next comment), fixed in 8898afd: on Windows without the symlink privilege, isolation now degrades cleanly to the previous render-in-place behavior.

[VincentGuyader]

False positive on both halves: no.. was added to list.files() in R 3.0.0 (r-source doc/NEWS.3: "list.files() (aka dir()) gains a new optional argument no.."), far older than the oldrel-1 leg (which passed green on this PR). And it is needed here: the call sites use all.files = TRUE, so removing no.. would feed . and .. into the symlink/copy loops.

[VincentGuyader]

Valid, fixed in 8898afd: every dir.create/file.symlink/file.copy/file.create return is now checked and stop()s on failure, so the tryCatch in render_single_qmd() sees a real error and the documented render-in-place fallback fires. A partially built isolated tree can no longer be rendered silently (it is also unlinked on failure).

[VincentGuyader]

Valid, and the highest-severity finding of the review (independently confirmed with an end-to-end reprex: a pre-existing doc.html was silently kept while the render reported success). Fixed in 8898afd: the snapshot is now an inventory of (path, mtime, size) and the copy-back takes files that are new or changed. A second-render regression test (real quarto render, edit, re-render, assert fresh content) locks it in.

[VincentGuyader]

Partially valid: the cleanup verb belongs to compile_qmd_course() (which performs clean_rendering_files()), and "exiting" was wrong on every path since the function returns FALSE. Note the isolated tree itself was cleaned by the finally. Fixed in 8898afd: the message is now just Failed to render {qmd} (<real error>).

[VincentGuyader]

Valid, with a sharper mechanism than stated: on Windows the failure is deterministic, not flaky (Sys.readlink() returns "" on platforms without the readlink syscall, so the nzchar() assertions are hard-red while the negative ones are vacuously green). Fixed in 8898afd with a capability probe (file.symlink + Sys.readlink round-trip in tempdir) and skip_if_not(), preferred over skip_on_os("windows") since restricted Linux mounts fail the same way.