feat: content-hash author-written asset urls so they cannot go stale

.agents/skills/webjs/references/built-ins.md

@@ -82,7 +82,7 @@ export const revalidate = 60;   // cache this page's HTML for 60s
 
 ### Content-hash asset URLs and conditional GET
 
-Both are automatic, prod-focused, and need no config. In production every served module and `public/` asset gets a per-file `?v=<hash>` and `Cache-Control: public, max-age=31536000, immutable`, so a returning client fetches a changed file only when its bytes change. Every cacheable response also carries a weak `ETag`, and a repeat request with a matching `If-None-Match` gets a `304 Not Modified` with no body. Unstorable (`no-store`) and streamed responses are excluded from the ETag path. A `private` response IS validated: `private` forbids SHARED storage, not validation, and the ETag hashes that response's own body, so two users with different bodies get different ETags and neither can match the other's, while two users with identical bodies are asking about identical bytes, where a 304 discloses nothing (#1140). That is what keeps the client router's partial responses cheap on a page that opted into caching; a default `no-store` page has nothing to validate either way. Dev is byte-faithful (no hashing).
+Both are automatic, prod-focused, and need no config. In production every served module gets a per-file `?v=<hash>`, and so does a `public/` asset you reference from markup: an `href` / `src` on a `<link>`, `<script>`, `<img>`, or `<source>` is rewritten at SSR whether the framework emitted it or you wrote it by hand (#1194). A `?v=`-carrying request is served `Cache-Control: public, max-age=31536000, immutable`, so a returning client fetches a changed file only when its bytes change, and a deploy cannot serve stale bytes at a live url. An asset reached some other way (a `url()` inside a stylesheet, a `srcset` candidate list, a url built at runtime in JS) keeps its plain path and the ~1h fallback, so give it a versioned name yourself if it must be immutable. Every cacheable response also carries a weak `ETag`, and a repeat request with a matching `If-None-Match` gets a `304 Not Modified` with no body. Unstorable (`no-store`) and streamed responses are excluded from the ETag path. A `private` response IS validated: `private` forbids SHARED storage, not validation, and the ETag hashes that response's own body, so two users with different bodies get different ETags and neither can match the other's, while two users with identical bodies are asking about identical bytes, where a 304 discloses nothing (#1140). That is what keeps the client router's partial responses cheap on a page that opted into caching; a default `no-store` page has nothing to validate either way. Dev is byte-faithful (no hashing).
 
 **A page's ETag is only useful if the page renders the same bytes twice.** The ETag is a hash of the response body, so any per-render-varying value anywhere in the document defeats it: a `Date.now()`, a `Math.random()`, an id from a module-scope counter (which never resets in a long-lived server), or a CSP nonce. The failure is silent and total. The page renders correctly, every content assertion still passes, the header is still present, and the only symptom is that no `If-None-Match` ever matches, so every revalidation ships the whole document instead of an empty 304. A page under CSP is excluded from the server HTML cache for exactly this reason (the nonce must differ per response). If a page opts into a public `Cache-Control`, guard it with a test that renders the page twice, through its layout, and asserts the two outputs are byte-identical.
 

framework-dev.md

@@ -49,7 +49,11 @@ The purge is zone-wide rather than a path list: all four hostnames (`webjs.dev`,
   cmp /tmp/o /tmp/e && echo fresh || echo stale
   ```
 
-The permanent fix for the assets themselves is a content hash in the url, the way the framework already fingerprints module imports and its own emitted urls (`?v=`, #243). `tailwind.css` and the brand marks are referenced by hand-written urls in `website/app/layout.ts` and `website/lib/design/brand.ts`, which that machinery never sees, so they stay on stable urls and depend on this purge. Fingerprinting them would retire the dependency; the workflow stays worthwhile as the safety net for anything else on a stable url.
+Author-written asset urls ARE content-hashed as of #1194, so the assets that caused those two incidents can no longer go stale. A `<link rel="stylesheet" href="/public/tailwind.css">` written by hand in a layout is served as `/public/tailwind.css?v=<hash>`, the same `?v=` treatment the framework already gave its own emitted urls and author-written module specifiers (#243, #369). New bytes therefore mean a new url, and `dev.js` already serves a `?v=`-carrying `/public/` request `immutable` for a year rather than the 1h fallback, so these assets got a large caching win alongside the correctness one.
+
+`versionAssetUrls` (in `packages/server/src/asset-hash.js`) runs from `buildDocumentParts` in `ssr.js`, the single seam every response path funnels through, and it matches `href` / `src` on `<link>` / `<script>` / `<img>` / `<source>` only. The tag-name anchoring is load-bearing, not incidental: this site renders CODE SAMPLES of markup, and a bare `src="…"` match would rewrite the code a reader is looking at. A highlighted sample escapes `<` and splits the token across spans, so a literal `<img` never appears inside one. Any change to that matcher must keep the code-sample test in `packages/server/test/importmap/version-asset-urls.test.js` passing.
+
+The purge workflow STAYS regardless. It is the safety net for anything still on a stable url (an asset referenced from CSS rather than markup, a `srcset` candidate list, which is deliberately not rewritten, or anything a future page emits outside those four tags), and it is what clears the edge when a deploy changes something the fingerprinter does not cover.
 
 ---
 

packages/server/src/asset-hash.js

@@ -447,3 +447,161 @@ export function versionModuleImports(source, importerAbs) {
   }
   return out + source.slice(last);
 }
+
+/**
+ * An asset-bearing element in the SSR'd document. Anchored on the literal tag
+ * name, which is what keeps a RENDERED CODE SAMPLE safe: the highlighter
+ * escapes `<` to `&lt;` (and splits the token across spans), so a sample that
+ * displays `<img src="/public/x.png">` never contains a literal `<img` and can
+ * never be matched here. That is the whole reason this matches tags rather
+ * than bare `src="…"` occurrences.
+ *
+ * The trailing guard is `(?![-\w])`, NOT `\b`. A word boundary treats `-` as a
+ * terminator, so `\b` also matched hyphenated CUSTOM ELEMENTS: `<img-zoom>`,
+ * `<link-preview>`, `<source-map>`. This framework is web-components-first, so
+ * those are ordinary component names, and their `src` / `href` is a reactive
+ * PROP, meaning author data. Rewriting it changed the value a component
+ * hydrates with, so `this.src.endsWith('.svg')` went false and an equality
+ * check against the path a server action returned stopped matching, and the
+ * value churned every deploy.
+ *
+ * @type {RegExp}
+ */
+const ASSET_TAG_RE = /<(?:link|script|img|source)(?![-\w])[^>]*>/gi;
+
+/**
+ * `href` / `src` inside an already-matched tag, either quote style. `srcset`
+ * is deliberately excluded: it is a comma-separated candidate list with
+ * descriptors, so it needs its own parse rather than a whole-value rewrite.
+ *
+ * @type {RegExp}
+ */
+const ASSET_ATTR_RE = /(\s(?:href|src)\s*=\s*)(["'])([^"']*)\2/gi;
+
+/**
+ * A `<link>` whose href is a HINT for a later request rather than the request
+ * itself. See the call site: versioning one desynchronizes it from whatever
+ * actually fetches the asset. `modulepreload` is listed for completeness, but
+ * those hrefs are framework-emitted and already carry `?v=`, so the
+ * already-queried guard skips them anyway.
+ *
+ * @type {RegExp}
+ */
+const PRELOAD_REL_RE = /\srel\s*=\s*["']?(?:preload|prefetch|modulepreload|dns-prefetch|preconnect)\b/i;
+
+/**
+ * Append `?v=<content-hash>` to same-origin asset urls an APP AUTHOR wrote by
+ * hand in a template, so they cannot serve stale bytes at a live url.
+ *
+ * The framework already fingerprints every url it emits itself (module
+ * specifiers, importmap targets, modulepreload hints), and `versionModuleImports`
+ * does the same for author-written module specifiers in served source. This
+ * closes the last gap: a `<link rel="stylesheet" href="/public/app.css">` or an
+ * `<img src="/public/logo.svg">` written in a layout.
+ *
+ * Why it matters beyond cache-busting: `dev.js` already serves a `?v=`-carrying
+ * `/public/*` request `immutable` for a year, while an un-versioned one gets the
+ * 1h fallback. So fingerprinting these urls both makes staleness structurally
+ * impossible (new bytes mean a new url) and upgrades them from a 1 hour cache to
+ * an immutable one. Without it, a CDN keeps serving the previous copy after a
+ * deploy until something purges it, which is exactly what shipped two visible
+ * regressions on webjs.dev in one day (#1179 then #1185).
+ *
+ * Per-url behaviour is `withAssetHash`, so every existing guard applies
+ * unchanged: a no-op when fingerprinting is disabled (dev, so dev output stays
+ * byte-identical), and for a cross-origin, protocol-relative, relative, or
+ * unresolvable url. A url that already carries a query is skipped here too,
+ * since it is author-controlled and may be meaningful.
+ *
+ * The base path is a parameter rather than module state on purpose: a setter
+ * that a call site forgets fails silently and only on a sub-path deploy, which
+ * is the hardest place to notice it. Same signature shape as `withAssetHash`.
+ *
+ * @param {string} html   the assembled document (or a fragment of it)
+ * @param {string} [basePath]  the active base path, already applied to the urls
+ * @returns {string}
+ */
+export function versionAssetUrls(html, basePath = '') {
+  if (!_enabled || !_appDir) return html;
+  if (typeof html !== 'string' || html.indexOf('<') === -1) return html;
+  return html.replace(ASSET_TAG_RE, (tag) => (
+    // A preload / prefetch hint does not FETCH the asset, it warms the cache
+    // for a request some OTHER consumer makes later, and the preload cache is
+    // keyed on the full url including the query. The obvious case is a font:
+    // the real request comes from `@font-face url()` inside the stylesheet,
+    // and CSS `url()` is deliberately out of scope here, so versioning the
+    // hint guarantees it can never match. The asset is then fetched TWICE and
+    // the browser logs "preloaded but not used", turning an optimization into
+    // a pessimization. Leave the hint on the url its consumer will request.
+    PRELOAD_REL_RE.test(tag) ? tag : tag.replace(ASSET_ATTR_RE, (whole, lead, quote, url) => {
+      // Leave an author-supplied query alone: appending to it risks changing
+      // a meaning we do not own, and `withAssetHash` would merge with `&`.
+      if (!url || url.indexOf('?') !== -1) return whole;
+      if (!isServablePublicUrl(url, basePath)) return whole;
+      // Split a fragment BEFORE hashing. `withAssetHash` appends `?v=` to
+      // whatever it is handed, so passing the whole thing would emit
+      // `/x.svg#logo?v=H`: the browser then requests `/x.svg` with no query
+      // (losing the immutable caching this exists for) AND the fragment
+      // becomes `logo?v=H`, which matches no element id. An SVG sprite
+      // (`#icon`) and a media fragment (`#t=10,20`) are both real, so the
+      // fragment is preserved and re-appended after the query.
+      const hashIdx = url.indexOf('#');
+      const path = hashIdx === -1 ? url : url.slice(0, hashIdx);
+      const frag = hashIdx === -1 ? '' : url.slice(hashIdx);
+      const versioned = withAssetHash(path, basePath);
+      if (versioned === path) return whole;
+      return `${lead}${quote}${versioned}${frag}${quote}`;
+    })
+  ));
+}
+
+/**
+ * Is this url one the STATIC-ASSET serve path would actually serve?
+ *
+ * This gate exists for security, not tidiness. `resolveUrlToFile` maps any
+ * same-origin path to `join(_appDir, …)` and only checks containment against
+ * `_appDir`, which is the PROJECT ROOT. That was harmless while
+ * `withAssetHash` saw framework-emitted urls exclusively, but an author's
+ * markup can carry ATTACKER-CONTROLLED data (`<img src=${user.avatarUrl}>`,
+ * a markdown image, a CMS-configured path). Without this check, such a value
+ * makes the renderer `readFileSync` + hash ANY file under the project root
+ * and publish the result: `<img src="/.env">` comes back as
+ * `/.env?v=<hash>`, leaking both that the file exists and a stable
+ * fingerprint of its exact bytes (so a rotation, or a guessed value, is
+ * detectable), for files the serve path deliberately 404s. It also turns one
+ * attribute into a synchronous read of an arbitrarily large file on the
+ * render path.
+ *
+ * So the gate is `public/` only. A path outside it is left untouched and never
+ * touches the filesystem, which is why the check is a pure string test run
+ * BEFORE any resolution.
+ *
+ * The three root-served paths (`/favicon.ico`, `/sw.js`, `/offline.html`) are
+ * deliberately NOT admitted, even though the serve path answers them. It
+ * REMAPS them into `public/` while `resolveUrlToFile` does not, so admitting
+ * them hashed the wrong file: with a stray `favicon.ico` at the project root,
+ * `/favicon.ico` was stamped with the hash of the ROOT file while the server
+ * served `public/favicon.ico`'s bytes, and stamped them `immutable` for a
+ * year. Changing the real icon then left the url byte-identical, pinning the
+ * stale copy indefinitely, which is worse than the 1h cache this replaces.
+ * Admitting them would need the remap replicated here, and they gain nothing:
+ * a page references its icon as `/public/favicon.svg` anyway.
+ *
+ * @param {string} url
+ * @param {string} basePath
+ * @returns {boolean}
+ */
+function isServablePublicUrl(url, basePath) {
+  if (url[0] !== '/' || url[1] === '/') return false;
+  let p = url;
+  if (basePath && p.startsWith(basePath + '/')) p = p.slice(basePath.length);
+  // Strip query / fragment before classifying, matching resolveUrlToFile.
+  const q = p.indexOf('?'); if (q !== -1) p = p.slice(0, q);
+  const h = p.indexOf('#'); if (h !== -1) p = p.slice(0, h);
+  let decoded = p;
+  try { decoded = decodeURIComponent(p); } catch { /* keep raw */ }
+  // A traversal segment can escape `public/` after `join` normalises it, and
+  // the serve path rejects exactly this, so refuse rather than resolve.
+  if (decoded.includes('..')) return false;
+  return decoded.startsWith('/public/');
+}

packages/server/src/ssr.js

@@ -3,7 +3,7 @@ import { resolve } from 'node:path';
 import { renderToString, isNotFound, isRedirect, isForbidden, isUnauthorized, lookupModuleUrl, isLazy, cspNonce } from '@webjsdev/core';
 import { importMapTag, vendorIntegrityFor, publishedBuildId, appSourceId, basePath, vendorPreconnectOrigins, vendorPreloadTargets } from './importmap.js';
 import { withBasePath } from './base-path.js';
-import { withAssetHash } from './asset-hash.js';
+import { withAssetHash, versionAssetUrls } from './asset-hash.js';
 import { jsonForScriptTag } from './script-tag-json.js';
 import { transitiveDeps, bareImports } from './module-graph.js';
 import { seedingEnabled, collectSeeds, buildSeedScript } from './action-seed.js';
@@ -157,7 +157,16 @@ export async function ssrPage(route, params, url, opts) {
     if (frameId && suspenseCtx.pending.length === 0) {
       const subtree = extractFrameSubtree(body, frameId);
       if (subtree !== null) {
-        const frameRes = htmlResponse(subtree, opts.status || 200, opts.req, url);
+        // Fingerprint authored asset urls here too (#1194). This branch slices
+        // the raw render output and returns BEFORE `buildDocumentParts`, so
+        // without this the same `<img>` inside a `<webjs-frame>` would ship
+        // versioned on a full page load and un-versioned on a frame nav. The
+        // frame swap would then overwrite the fingerprinted attribute in the
+        // live DOM with the stable url, dropping it back to the short cache
+        // and re-opening exactly the staleness this change closes. It also
+        // keeps frame-render.js's byte-equivalence invariant (the subtree
+        // matches what the client would have extracted from the full page).
+        const frameRes = htmlResponse(versionAssetUrls(subtree, basePath()), opts.status || 200, opts.req, url);
         // The subtree is sliced by the x-webjs-frame REQUEST header, so a
         // shared cache must never serve it to a request that did not send
         // one (the same #1009 poisoning shape as the reduced-have case).
@@ -1176,6 +1185,17 @@ function buildHeadInner(opts) {
  * @returns {{ prefix: string, streamBody: string, closer: string }}
  */
 function buildDocumentParts(body, wrapOpts) {
+  // Content-hash asset urls the APP AUTHOR wrote by hand (#1194). Applied here
+  // rather than at each response site so every caller (buffered, streamed, and
+  // `buildDocument`) is covered by one seam. A pure no-op in dev and whenever
+  // fingerprinting is off, so output stays byte-identical there.
+  //
+  // The basePath is applied to author urls by the app itself (an author writes
+  // the url they want served), so it is passed only for FILE RESOLUTION, the
+  // same way `withAssetHash` uses it.
+  const bp = basePath();
+  const fpAssets = (s) => versionAssetUrls(s, bp);
+
   const shell = extractUserShell(body);
   if (shell) {
     const headInner = buildHeadInner(wrapOpts);
@@ -1187,12 +1207,12 @@ function buildDocumentParts(body, wrapOpts) {
       `<!doctype html>\n<html${shell.htmlAttrs}>\n<head${shell.headAttrs}>\n` +
       composedHead +
       `\n</head>\n<body${shell.bodyAttrs}>\n`;
-    return { prefix, streamBody: hoist.body, closer: `\n</body>\n</html>` };
+    return { prefix: fpAssets(prefix), streamBody: fpAssets(hoist.body), closer: `\n</body>\n</html>` };
   }
   // No user shell: framework owns the wrapper.
   const headHtml = wrapHead(wrapOpts);
   const { head, body: bodyOut } = hoistHeadTags(headHtml, body);
-  return { prefix: head, streamBody: bodyOut, closer: `\n</body>\n</html>` };
+  return { prefix: fpAssets(head), streamBody: fpAssets(bodyOut), closer: `\n</body>\n</html>` };
 }
 
 // Re-export for unit testing.
@@ -2121,8 +2141,13 @@ function streamingHtmlResponse(prefix, bodyHtml, closer, ctx, status, req, url,
             // <script> is here for legacy / extremely-restrictive
             // environments. Either way it must be nonce-signed.
             const scriptNonce = nonce ? ` nonce="${escapeAttr(nonce)}"` : '';
+            // Fingerprint authored asset urls here too (#1194). A streamed
+            // boundary is enqueued directly and never passes through
+            // `buildDocumentParts`, so without this the SAME `<img>` would get
+            // `?v=` in the shell but not inside a Suspense boundary, which is
+            // a difference no author would predict from where they wrote it.
             const chunk =
-              `<template data-webjs-resolve="${r.id}">${r.html}</template>` +
+              `<template data-webjs-resolve="${r.id}">${versionAssetUrls(r.html, basePath())}</template>` +
               `<script${scriptNonce}>window.__webjsResolve&&__webjsResolve("${r.id}")</script>`;
             controller.enqueue(encoder.encode(chunk));
           }