fix!: v0.4.0 bug batch — fences, whitespace, type safety, @extends FM (#149–#154)

.devflow/docs/design/149-154-v0.4.0-breaking-fixes.2026-07-08_1400.md

@@ -0,0 +1,118 @@
+---
+issues: [149, 150, 151, 152, 153, 154]
+execution-strategy: SINGLE_CODER
+date: 2026-07-08
+---
+
+# v0.4.0 Breaking Fixes: Issues #149–#154
+
+Six approved contracts implemented as 6 sequential commits, each fully green
+before committing (`cargo test --workspace` + `cargo fmt --all --check` +
+`cargo clippy --workspace --all-targets -- -D warnings`).
+
+This plan deliberately amends ADR-001's R3 rule — R3 (blank-line-run capping)
+is removed because the new whitespace contract (#150/#151) makes it unsound.
+The formatter is re-pitched as a line-ending/trailing-whitespace normalizer.
+
+## Approved Contracts
+
+### Contract 1 — Whitespace = "delete-directive-lines" model (#150, #151)
+
+A directive line consumes exactly its own line + trailing newline; every other
+markdown-mode byte passes through verbatim (leading blanks, interior blank runs
+uncapped, post-frontmatter blank).
+
+Carve-outs:
+- Trailing edge still normalizes to exactly one final `\n` (whitespace-only
+  doc → `""`).
+- `\r` still stripped.
+- `@message`/`@define` bodies keep an eval-time edge `.trim()` (interior
+  verbatim).
+
+Docs call it "interior-verbatim with trailing-edge normalization" — never bare
+"byte-exact".
+
+### Contract 2 — Fences, full scope (#149)
+
+Recognition prefix widened to `[ \t>]*` — any indentation depth, tabs
+(byte-counted), blockquote markers (`>`, nested `> >`). Tilde fences (`~~~`)
+also recognized. Fence lines preserved verbatim in output. Unclosed-fence error
+gains the opener's span.
+
+Out of scope (documented): 4-space indented code blocks without fence markers;
+implicit fence close by leaving a blockquote.
+
+### Contract 3 — Cross-type comparisons + --set-string (#152)
+
+Keep `--set` coercion. ANY cross-type `==`/`!=` is a hard eval-time error
+(`mds::type_mismatch`, line-level span, fix-it hint). Same-type Array/Object
+comparisons become structural equality. And/Or stays lazy (skipped operand
+never errors). New `--set-string` on build/check/watch. Duplicate key across
+`--set`/`--set-string` = hard error.
+
+### Contract 4 — @extends emits deep-merged frontmatter (#154)
+
+`@extends` emits the deep-merged frontmatter (child wins, transitive).
+Knowingly reverses the "base FM is scope-only" posture (f4 `base_secret` test).
+Requires explicit-intent test rewrite, loud BREAKING changelog, adversarial
+YAML-injection tests.
+
+Standalone path stays raw-verbatim (named asymmetry). Reserved keys
+(`type`/`imports`/`extends`) stripped top-level ONLY, uniformly. Child-without-
+frontmatter now emits base keys; empty merged mapping → no fence. Emitted FM
+ignores `--set` (documented).
+
+### Contract 5 — Interpolation hint fix (#153)
+
+`parser_helpers.rs:1102` renders `\{{` (broken); must render `\{`.
+Fix: format string `\\{{{{` → `\\{{`.
+
+## Six-Commit Breakdown
+
+### Commit 1 — `fix: interpolation error hint recommends \{ (#153)`
+Files: `crates/mds-core/src/parser_helpers.rs`
+- Fix format string at line ~1102: `\\{{{{` → `\\{{`
+- Add test pinning the rendered hint text
+
+### Commit 2 — `fix!: interior-verbatim whitespace contract (#150, #151)` — KEYSTONE
+Files: `parser.rs`, `parser_helpers.rs`, `lib.rs`, `formatter.rs`, `evaluator.rs`
+- Delete `strip_trailing_newline(strip_leading_newline(body))` at 3 parser sites
+- Delete dead `strip_leading_newline`/`strip_trailing_newline` functions
+- Add `.trim()` at `invoke_function` body eval (line ~376)
+- Rewrite `clean_output`: keep \r strip + trailing trim_end() + one \n + ws-only→"";
+  DELETE leading trim_start_matches, run-cap + newline_count
+- R3 death in formatter: delete `protected_spans()`, leading_mode, newline_run,
+  R3 blank-line logic; `rewrite_body` → 3 branches: directive→R4 | raw-content→verbatim | else→R1-only
+- Update all affected tests
+
+### Commit 3 — `fix: recognize indented, tilde, and blockquoted code fences (#149)`
+Files: `crates/mds-core/src/lexer.rs`
+- New `fence_at_line_start()` helper: scan `[ \t>]*` then require ≥3 backtick or `~`
+- State: `code_fence_backticks: usize` → `code_fence: Option<(char, usize)>` + opener span
+- Closer: verbatim capture (not reconstructed)
+- Unclosed-fence error gains opener span
+- Tests: lexer unit + fmt corpus + integration repro
+
+### Commit 4 — `fix!: cross-type comparison errors, structural equality, --set-string (#152)`
+Files: `evaluator.rs`, `error.rs`, `crates/mds-cli/src/main.rs`, `build.rs`, `watch.rs`
+- New `values_equal_same_type()` returning `Option<bool>` (None = cross-type error)
+- New `MdsError::TypeMismatch` variant with `mds::type_mismatch` code
+- Add `file`/`source` to `EvalContext`
+- New `RuntimeVarArgs` struct (flatten into Build/Check/Watch)
+- `--set-string` flag: inserts as `Value::String`, bypasses coercion
+- Tests: flip 3 existing, add cross-type matrix + structural eq + lazy short-circuit
+
+### Commit 5 — `fix!: @extends emits deep-merged frontmatter (#154)`
+Files: `resolver/frontmatter.rs`, `resolver.rs`, `lib.rs`
+- `serialize_merged_frontmatter()` helper
+- `build_merged_extends_scope()` returns `(Scope, Option<String>)` 
+- Extends emit site switches to `prepend_frontmatter(merged_fm, body)`
+- Reserved keys gate on `depth == 0`
+- Rewrite f4 test; augment f6; add adversarial YAML-injection tests
+
+### Commit 6 — `docs: v0.4.0 language/CLI docs, changelog, examples`
+Files: `spec.md`, `README.md`, `CHANGELOG.md`, `.devflow/features/mds-fmt/KNOWLEDGE.md`, `examples/`
+- Update spec.md whitespace contract section
+- Update README.md `mds fmt` pitch (remove "collapses blank lines")
+- CHANGELOG.md: Fixed/Changed-BREAKING/Added entries for #149-#154
+- Re-sync mds-fmt KNOWLEDGE.md (R3 gone, clean_output new behavior)

.github/workflows/ci.yml

@@ -94,12 +94,16 @@ jobs:
             gz=$(gzip -c "$f" | wc -c | tr -d ' ')
             label="${f#crates/mds-wasm/}"
             echo "::notice::WASM ${label}: ${raw} bytes raw, ${gz} bytes gzipped"
-            # Soft bloat guard. Budget raised 600K->650K (2026-06): the unpinned CI
-            # `stable` toolchain drifted and pushed the optimized binary just over 600K
-            # with no source change (local toolchain still emits ~592K). Follow-up: pin
-            # the wasm build toolchain to make the size deterministic and re-tighten this.
-            if [ "$raw" -gt 650000 ]; then
-              echo "::error::WASM binary exceeds 650,000 byte threshold: ${label} is ${raw} bytes"
+            # Soft bloat guard. Budget history:
+            #   600K initial; raised 600K->650K (2026-06): unpinned CI `stable` toolchain
+            #   drifted past 600K with no source change (local emitted ~592K).
+            #   raised 650K->700K (2026-07, PR #161 / fix/v0-4-0-149-154): toolchain
+            #   drift + new v0.4.0 code (cross-type errors, extended help strings,
+            #   @extends frontmatter merging) pushed CI binary to 650,745 bytes.
+            # Follow-up: pin the wasm build toolchain to make the size deterministic
+            # and re-tighten this guard.
+            if [ "$raw" -gt 700000 ]; then
+              echo "::error::WASM binary exceeds 700,000 byte threshold: ${label} is ${raw} bytes"
               exit 1
             fi
           done

CHANGELOG.md

@@ -7,21 +7,50 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### **BREAKING** — Strict cross-type comparisons, merged `@extends` frontmatter, interior-verbatim whitespace
+
+These changes alter observable runtime behavior and compiled output. Templates relying
+on the previous (buggy) behavior must be updated.
+
+#### Cross-type comparisons are now errors (#152)
+
+`@if a == b:` or `@if a != b:` where `a` and `b` are different types (e.g. a number
+vs. a string, or a boolean vs. null) now raises `mds::type_mismatch` at runtime
+instead of silently returning `false` (for `==`) or `true` (for `!=`).
+
+**Migration:** add an explicit conversion before comparing:
+- `@if str(count) == "3":` — convert number to string
+- `@if count == 3:` — compare number to number literal
+
+#### `@extends` emits deep-merged frontmatter (#154)
+
+Compiled output for a child template now contains the **deep-merged** frontmatter
+(base keys + child keys, child wins on collision, reserved keys `imports`/`type`/`extends`
+excluded) rather than only the child's raw frontmatter. Base-only frontmatter keys
+now appear in the compiled output.
+
+**Migration:** if your pipeline depends on base frontmatter keys being absent from the
+compiled output, strip them downstream or move them to a non-frontmatter location.
+
 ### Added
 
+- **`--set-string KEY=VALUE`** CLI flag for `mds build`, `mds check`, and `mds watch`.
+  Sets a variable as a string without type coercion — useful when a value is
+  numeric-looking but must stay a string (e.g. `mds build t.mds --set-string id=007`).
+  Repeatable. (#152)
+
 - **`mds fmt`** — an opinionated, safety-gated auto-formatter for `.mds` templates. Every
   rewrite is guaranteed compile-equivalent: a runtime safety gate re-compiles the formatted
   source and refuses to write if it would change compiled output (`mds::formatter_invariant`)
   rather than silently corrupting a template. Normalizes CRLF to LF everywhere (including
-  inside frontmatter and code fences), collapses runs of two or more consecutive blank lines
-  to a single blank line, strips trailing whitespace on directive lines, and ensures exactly
-  one final newline — while leaving body-text trailing whitespace (Markdown hard breaks),
-  blank-line structure within frontmatter and code fences, and the byte-for-byte content of
-  `@message`/`@define` bodies untouched. Supports a single file,
-  a directory (recursive, including `_`-prefixed partials), or stdin (`-`, as a filter);
-  `--check` exits non-zero without writing when anything would change, and `--diff` prints a
-  unified diff (colorized on a TTY) without writing. New public `mds-core` API:
-  `format_str` / `format_str_with`. (#60)
+  inside frontmatter and code fences), strips trailing whitespace on directive lines, and
+  ensures exactly one trailing newline — while leaving interior blank lines, blank-line
+  structure within frontmatter and code fences, body-text trailing whitespace (Markdown
+  hard breaks), and the byte-for-byte content of `@message`/`@define` bodies untouched.
+  Supports a single file, a directory (recursive, including `_`-prefixed partials), or
+  stdin (`-`, as a filter); `--check` exits non-zero without writing when anything would
+  change, and `--diff` prints a unified diff (colorized on a TTY) without writing. New
+  public `mds-core` API: `format_str` / `format_str_with`. (#60)
 
 - **Native Python bindings** (`crates/mds-python`, PyO3 + maturin), to be distributed
   as `mdscript` on PyPI. Seven functions — `compile`, `compile_file`,
@@ -38,6 +67,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- **BREAKING:** Interior-verbatim whitespace contract for block bodies and `mds fmt`.
+  Leading blank lines and interior blank runs inside `@block` / `@define` bodies and
+  `mds fmt` output are now preserved verbatim; previously they were collapsed or stripped.
+  The `mds fmt` blank-line collapsing rule (R3) has been removed to maintain compile
+  equivalence with the updated evaluator behavior. Only the trailing edge normalizes (to
+  exactly one final newline). **Migration:** compiled outputs may gain blank lines that
+  were previously collapsed or stripped; templates relying on this collapse must remove
+  the extra blank lines at the source level. (#150, #151)
+
 - **BREAKING:** `FileSystem` trait now requires two new methods — `normalize_in_dir`
   and `parent_dir` — that replace the internal `<source>` path-sentinel pattern.
   String-source `@import`/`@extends` resolution is now directly directory-anchored:
@@ -48,6 +86,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- **Code fences: tilde (`~~~`), indented, and blockquoted variants are now recognized**
+  as passthrough regions. Previously only `` ``` ``-fences that started at column 1
+  were treated as code — a `~~~` fence, a `` > ``` `` blockquote fence, or a fence
+  indented with spaces/tabs would allow interpolation and directive parsing inside,
+  silently corrupting output for affected templates. The lexer now matches any fence
+  that starts with `[ \t>]*` followed by three or more matching backticks or tildes.
+  (#149)
+
+- **Interpolation errors now suggest `\{`** in the help text when an opening brace
+  appears in a context where an interpolation was expected but failed (e.g. `{ invalid`
+  or an unclosed brace). Helps users writing literal brace characters. (#153)
+
 - **Windows: string-source `@import`/`@extends` now resolve relative imports
   correctly.** `std::fs::canonicalize` returns a `\\?\` verbatim extended-length path
   on Windows, and inside a `\\?\` prefix `/` is a literal character, not a path

README.md

@@ -87,7 +87,8 @@ Build/Watch options:
   --out-dir <DIR>             Output directory (build/single-file watch: <stem>.md or <stem>.json;
                               dir-mode watch: mirrors source subtree)
   --vars <FILE>               JSON file with variable overrides (reloaded each rebuild)
-  --set KEY=VALUE             Set a single variable (repeatable)
+  --set KEY=VALUE             Set a single variable (repeatable); value coerced to number/bool/null/array when possible
+  --set-string KEY=VALUE      Set a single variable as a string, bypassing type coercion (repeatable)
 
 Watch-only options:
   --clear                     Clear terminal before each rebuild (only when stderr is a TTY)
@@ -172,20 +173,17 @@ echo 'Hello   {name}!' | mds fmt -   # format from stdin, write to stdout; creat
 What it normalizes:
 
 - CRLF → LF, everywhere (including inside frontmatter and code fences)
-- Two or more consecutive blank lines collapse to a single blank line; leading blank lines in the body are removed entirely
 - Trailing whitespace on `@if`/`@for`/`@define`/… directive lines is stripped
 - Exactly one final newline (empty or whitespace-only input formats to an empty file)
 
 What it deliberately leaves untouched:
 
-- Trailing whitespace on body-text content lines, including whitespace-only "blank" lines — two
-  trailing spaces are a Markdown hard line break, and stripping them (anywhere in body text,
-  including a stray blank line) can change rendered output
-- Blank-line structure within frontmatter and code fences — CRLF is normalized there (the same
-  as everywhere else in the file), but blank lines are not collapsed inside these regions
-- The byte-for-byte content of `@message`/`@define` bodies — neither CRLF normalization nor
-  blank-line collapsing is applied inside them, because these bodies bypass the compiler's own
-  whitespace normalization and their content reaches compiled output verbatim
+- Trailing whitespace on body-text content lines — two trailing spaces are a Markdown hard line
+  break; stripping them would change rendered output
+- Blank-line structure within the file body, frontmatter, and code fences — the formatter does
+  not add or remove blank lines; blank-line layout is the template author's choice
+- The byte-for-byte content of `@message`/`@define` bodies — whitespace inside these bodies
+  reaches compiled output verbatim and must not be altered
 
 Directory mode formats every `.mds` file recursively, **including `_`-prefixed partials**,
 continuing past per-file errors and printing a summary
@@ -269,7 +267,7 @@ live in [`examples/`](examples/).
 
 ## Language Reference
 
-See [spec.md](spec.md) for the full MDS v0.2.0 language specification.
+See [spec.md](spec.md) for the full MDS v0.4.0 language specification.
 
 ## Contributing