fix(seo): make lastmod describe the page, not the row behind it

[MarkusNeusinger]

Measured, not guessed

Search Console, queried just now:

sitemap last read                  2026-08-14      before all of today's work
/heatmap-annotated/python/altair   crawled 2026-07-29   three weeks ago
/bar-error/python/matplotlib       crawled 2026-08-07
/                                  crawled 2026-08-17

Everything shipped today is invisible to anything grounding on Google's index. That is precisely the symptom reported from live testing: Mistral works, Gemini does not — Mistral fetches the page, Gemini reads Google's copy, and Google's copy is weeks old.

The sitemap was arguing against a recrawl

lastmod came from an implementation's updated column. Today changed the rendering, not the data — real render markup, both themes, the interactive version, a rewritten meta description — so every date stayed exactly where it was, and the sitemap reported that nothing had changed.

lastmod is the one signal that asks a crawler to come back, and it was saying don't bother.

Fix

lastmod is now the later of the record's own date and TEMPLATE_LAST_CHANGED.

TEMPLATE_LAST_CHANGED = datetime(2026, 8, 18)

Bump it only when the rendered page genuinely changes for every URL. It asserts to search engines that ~3,900 pages changed at once; making that claim casually is how a site teaches Google to ignore its lastmod altogether. The constant carries that warning in a comment, and the docs repeat it.

A missing updated no longer yields a missing lastmod either — the page was last modified at least when its template was, and saying nothing was the same lost signal in a smaller form.

Verification

• pytest tests/unit — 1658 passed. Three TestLastmod cases now pin the contract explicitly: a newer record wins, an older one is lifted, and no record still yields the template date
• ruff check + ruff format --check — clean
• Generated output inspected directly: a spec dated 2099 keeps its own date, an implementation dated 2025 gets 2026-08-18

Needs a manual step to take effect

The sitemap is only re-read every few days on its own. Resubmitting it in Search Console (Sitemaps → the existing entry → resubmit) pairs with this bump and is the fastest way to get the recrawl started. Same page also offers URL Inspection → Request indexing for a handful of priority URLs, capped around ten a day.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Updates sitemap <lastmod> generation so it reflects when the rendered page template changed (not only when DB rows changed), improving recrawl signaling for search engines and bots.

Changes:

• Introduces TEMPLATE_LAST_CHANGED and uses max(record.updated, TEMPLATE_LAST_CHANGED) to compute sitemap <lastmod>.
• Updates unit tests to pin the new lastmod contract (newer record wins; older/None record yields template date).
• Documents the new lastmod semantics and adds a changelog entry.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
api/routers/seo.py	Adds TEMPLATE_LAST_CHANGED and updates _lastmod() to prefer the later of record date vs template date.
tests/unit/api/test_seo_helpers.py	Updates/extends tests to assert the new <lastmod> behavior.
docs/reference/seo.md	Documents the “page vs row” lastmod rule and when to bump the template constant.
CHANGELOG.md	Adds an Unreleased “Fixed” entry describing the sitemap recrawl signal correction.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[MarkusNeusinger]

Both applied — prose outrunning the code in each case.

The _lastmod docstring was left over from an earlier draft that did return empty; it cannot any more, since the template date is always defined. That is the point of the change and the tests assert it, so the docstring was the only thing still claiming otherwise.

And the sitemap section did describe only the implementation tier while hub URLs use spec.updated. Now says "the record's", naming both.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

api/routers/seo.py:45

• _lastmod ultimately emits a YYYY-MM-DD string, but it currently compares full datetime objects via max(dt, TEMPLATE_LAST_CHANGED). Comparing datetimes is unnecessary here and can fail if dt is ever timezone-aware (offset-aware vs naive comparisons raise TypeError). Consider storing TEMPLATE_LAST_CHANGED as a date and comparing dates only, then emit via isoformat().

TEMPLATE_LAST_CHANGED = datetime(2026, 8, 18)


def _lastmod(dt: datetime | None) -> str:
    """Format the later of the record's own date and the template's, as <lastmod>.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!