fix(ci): the bot-serving monitor has been failing for ten days - #10478
Merged
Conversation
Every scheduled run since 2026-08-09 failed, on one line: the check greps
for the literal `<title>anyplot.ai</title>` and the home page now says
"anyplot.ai — AI-generated plot catalog for 15 libraries". Nothing else
was broken. Nobody looked.
That is the uncomfortable part. This run of SEO work edited this workflow
twice, and cited it repeatedly as the regression cover for exactly the
class of change that has no local verification loop — nginx behaviour,
bot serving. It was red the whole time. A monitor nobody reads protects
nothing, and the trailing-slash port leak that shipped today is the proof:
the guard was there and it was already failing.
The home check now matches on the prefix. That still distinguishes the
prerendered page from the SPA shell, whose title is "any.plot() — any
library.", which is the property under test; the full title was never the
point and made the check break on copy edits.
The trailing-slash case also no longer passes when there is no redirect
at all. An empty %{redirect_url} fell through to the success branch and
printed "OK: trailing slash -> " — so the rewrite disappearing entirely
would have read as a pass. Copilot raised this on two separate PRs and
neither addressed it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a flaky/brittle GitHub Actions synthetic monitor (bot-serving-check) that validates the “bot → seo-proxy” serving path and trailing-slash redirect behavior for anyplot.ai. It updates the monitor to be resilient to homepage title copy changes and to correctly fail when the trailing-slash redirect disappears.
Changes:
- Relax the homepage
<title>assertion to a stable prefix (<title>anyplot.ai) so copy edits don’t break the monitor. - Treat an empty
%{redirect_url}for the trailing-slash probe as a failure (indicating no redirect). - Add a changelog entry documenting the monitor fix and why it mattered.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
CHANGELOG.md |
Adds an [Unreleased] “Fixed” entry describing the monitor regression and the remediation. |
.github/workflows/bot-serving-check.yml |
Makes the homepage title check robust to copy changes and hardens the trailing-slash redirect assertion. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+241
to
+243
| under test — and no longer breaks on copy. The trailing-slash check also no longer passes when | ||
| there is no redirect at all: an empty `%{redirect_url}` printed "OK" and returned success, which | ||
| Copilot raised on two separate PRs (#10478). |
Comment on lines
113
to
+117
| slash_target=$(curl -sS --max-time 30 -o /dev/null -A "$GOOGLEBOT" \ | ||
| -w '%{redirect_url}' "$ORIGIN/scatter-basic/") | ||
| case "$slash_target" in | ||
| "") | ||
| # No redirect at all: %{redirect_url} is empty, which the previous |
MarkusNeusinger
added a commit
that referenced
this pull request
Aug 18, 2026
Three findings from a fresh-context audit of today's twelve PRs. I had spotted none of them. ## 1. The documentation contradicted itself `docs/reference/seo.md`'s bot-map section still said `gptbot`, `meta-externalagent` and `amazonbot` were *"declined in robots.txt"*, and `app/nginx.conf` repeated the claim in a comment. Both were written about an hour before #10474 opened the policy, and neither was reconciled — so the page asserted the old policy three screens from the section declaring the new one. The measured edge-state table was stale the other way round: the dashboard unblock it prescribed had since been carried out, so the table described a state that no longer existed. It now records what is actually blocked (`Bytespider`, `TikTok Spider`, and three agents whose rule-compliance is unverified rather than disproven) and says plainly that `bot-serving-check` tests the origin and will never catch edge drift. ## 2. A database outage would reopen #10453 With no catalogue to check against, the bot routes answered `200` with a fabricated, self-canonicalising page for **any** string — the precise defect #10453 removed, surviving in degraded mode. Degraded pages now carry `noindex`. I first tried returning `503`, which is arguably more correct, and backed it out: it broke eleven tests that use the no-DB path as a rendering harness. `noindex` keeps the behaviour those tests depend on and removes the indexing risk, which is the part that matters. The path is unreachable in production — but "unreachable" here means one misconfiguration away from indexable. ## 3. 404s were counted as successful page reads `bot_fetch` ran as a router dependency. A dependency executes **before** the handler and cannot see the response, so every miss was recorded as a read. It has moved to a middleware and gained a `status` property. Recording the miss is right — an assistant asking for a URL that no longer exists is how a library migration announces itself — but recording it as a page view is a lie. Filter on `status` before reading anything else; documented in `docs/reference/plausible.md`. ## Also A docstring pointed at `app/src/router.tsx`, which does not exist. Routing lives in `app/src/routes/index.tsx`. ## Verification - `pytest tests/unit` — 1640 passed, including: degraded hub and impl pages assert `noindex`, a companion test asserts normal pages do **not**, and a `bot_fetch` test pins `status: "404"` on a miss - `ruff check` + `ruff format --check` — clean - `grep` confirms no remaining reference to the superseded policy in `seo.md`, `nginx.conf` or `robots.txt` ## Not in this PR The audit's other findings are handled elsewhere: the ten-day-red monitor in #10478, and the analytics that recorded nothing in #10477. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The finding
Ten consecutive scheduled runs, all red, on one line:
Nothing else was broken.
grep -qFon the full title, copy changed, check died.Why it matters more than a stale string
This run of SEO work edited this workflow twice and cited it repeatedly as the regression cover for precisely the class of change that has no local verification loop — nginx behaviour, bot serving. It was failing daily throughout.
The trailing-slash port leak that shipped today is the proof: the guard existed, it would have caught it, and it was already red so nobody would have seen it. A monitor nobody reads protects nothing.
Found by an independent audit, not by me — I had asserted the guard's value in three PR bodies without once checking whether it was green.
Changes
Home check matches on the prefix.
<title>anyplot.aistill separates the prerendered page from the SPA shell, whose title isany.plot() — any library.— that separation is the property under test. The full title was never the point and made the check break on copy edits.Empty redirect is now a failure.
%{redirect_url}is empty when there is no redirect, which fell through to the success branch and printedOK: trailing slash ->. So the rewrite disappearing entirely would have read as a pass. Copilot raised this on #10473 and again on #10476; neither addressed it.After merge
The workflow needs a manual
workflow_dispatchto confirm the first green run in ten days — the schedule alone would leave it unverified until tomorrow. Note it will only pass once #10476 has deployed, since the trailing-slash assertion is currently correct to fail.Not fixed here
Nothing alerted on ten days of failure. The workflow reports into GitHub Actions and that is the whole notification path. Worth a decision separately — a monitor that fails silently is only marginally better than none.
🤖 Generated with Claude Code