Skip to content

fix(ci): the bot-serving monitor has been failing for ten days - #10478

Merged
MarkusNeusinger merged 1 commit into
mainfrom
fix/ci-bot-serving-check-has-been-red
Aug 18, 2026
Merged

fix(ci): the bot-serving monitor has been failing for ten days#10478
MarkusNeusinger merged 1 commit into
mainfrom
fix/ci-bot-serving-check-has-been-red

Conversation

@MarkusNeusinger

Copy link
Copy Markdown
Owner

The finding

$ gh run list --workflow=bot-serving-check.yml --limit 10
2026-08-18  failure (schedule)
2026-08-17  failure (schedule)
2026-08-16  failure (schedule)
...
2026-08-09  failure (schedule)

Ten consecutive scheduled runs, all red, on one line:

expects: <title>anyplot.ai</title>
actual:  <title>anyplot.ai — AI-generated plot catalog for 15 libraries</title>

Nothing else was broken. grep -qF on the full title, copy changed, check died.

Why it matters more than a stale string

This run of SEO work edited this workflow twice and cited it repeatedly as the regression cover for precisely the class of change that has no local verification loop — nginx behaviour, bot serving. It was failing daily throughout.

The trailing-slash port leak that shipped today is the proof: the guard existed, it would have caught it, and it was already red so nobody would have seen it. A monitor nobody reads protects nothing.

Found by an independent audit, not by me — I had asserted the guard's value in three PR bodies without once checking whether it was green.

Changes

Home check matches on the prefix. <title>anyplot.ai still separates the prerendered page from the SPA shell, whose title is any.plot() — any library. — that separation is the property under test. The full title was never the point and made the check break on copy edits.

Empty redirect is now a failure. %{redirect_url} is empty when there is no redirect, which fell through to the success branch and printed OK: trailing slash -> . So the rewrite disappearing entirely would have read as a pass. Copilot raised this on #10473 and again on #10476; neither addressed it.

After merge

The workflow needs a manual workflow_dispatch to confirm the first green run in ten days — the schedule alone would leave it unverified until tomorrow. Note it will only pass once #10476 has deployed, since the trailing-slash assertion is currently correct to fail.

Not fixed here

Nothing alerted on ten days of failure. The workflow reports into GitHub Actions and that is the whole notification path. Worth a decision separately — a monitor that fails silently is only marginally better than none.

🤖 Generated with Claude Code

Every scheduled run since 2026-08-09 failed, on one line: the check greps
for the literal `<title>anyplot.ai</title>` and the home page now says
"anyplot.ai — AI-generated plot catalog for 15 libraries". Nothing else
was broken. Nobody looked.

That is the uncomfortable part. This run of SEO work edited this workflow
twice, and cited it repeatedly as the regression cover for exactly the
class of change that has no local verification loop — nginx behaviour,
bot serving. It was red the whole time. A monitor nobody reads protects
nothing, and the trailing-slash port leak that shipped today is the proof:
the guard was there and it was already failing.

The home check now matches on the prefix. That still distinguishes the
prerendered page from the SPA shell, whose title is "any.plot() — any
library.", which is the property under test; the full title was never the
point and made the check break on copy edits.

The trailing-slash case also no longer passes when there is no redirect
at all. An empty %{redirect_url} fell through to the success branch and
printed "OK: trailing slash -> " — so the rewrite disappearing entirely
would have read as a pass. Copilot raised this on two separate PRs and
neither addressed it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 18, 2026 19:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a flaky/brittle GitHub Actions synthetic monitor (bot-serving-check) that validates the “bot → seo-proxy” serving path and trailing-slash redirect behavior for anyplot.ai. It updates the monitor to be resilient to homepage title copy changes and to correctly fail when the trailing-slash redirect disappears.

Changes:

  • Relax the homepage <title> assertion to a stable prefix (<title>anyplot.ai) so copy edits don’t break the monitor.
  • Treat an empty %{redirect_url} for the trailing-slash probe as a failure (indicating no redirect).
  • Add a changelog entry documenting the monitor fix and why it mattered.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
CHANGELOG.md Adds an [Unreleased] “Fixed” entry describing the monitor regression and the remediation.
.github/workflows/bot-serving-check.yml Makes the homepage title check robust to copy changes and hardens the trailing-slash redirect assertion.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread CHANGELOG.md
Comment on lines +241 to +243
under test — and no longer breaks on copy. The trailing-slash check also no longer passes when
there is no redirect at all: an empty `%{redirect_url}` printed "OK" and returned success, which
Copilot raised on two separate PRs (#10478).
Comment on lines 113 to +117
slash_target=$(curl -sS --max-time 30 -o /dev/null -A "$GOOGLEBOT" \
-w '%{redirect_url}' "$ORIGIN/scatter-basic/")
case "$slash_target" in
"")
# No redirect at all: %{redirect_url} is empty, which the previous
@MarkusNeusinger
MarkusNeusinger merged commit c8e2917 into main Aug 18, 2026
8 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the fix/ci-bot-serving-check-has-been-red branch August 18, 2026 19:57
MarkusNeusinger added a commit that referenced this pull request Aug 18, 2026
Three findings from a fresh-context audit of today's twelve PRs. I had
spotted none of them.

## 1. The documentation contradicted itself

`docs/reference/seo.md`'s bot-map section still said `gptbot`,
`meta-externalagent` and `amazonbot` were *"declined in robots.txt"*,
and `app/nginx.conf` repeated the claim in a comment. Both were written
about an hour before #10474 opened the policy, and neither was
reconciled — so the page asserted the old policy three screens from the
section declaring the new one.

The measured edge-state table was stale the other way round: the
dashboard unblock it prescribed had since been carried out, so the table
described a state that no longer existed. It now records what is
actually blocked (`Bytespider`, `TikTok Spider`, and three agents whose
rule-compliance is unverified rather than disproven) and says plainly
that `bot-serving-check` tests the origin and will never catch edge
drift.

## 2. A database outage would reopen #10453

With no catalogue to check against, the bot routes answered `200` with a
fabricated, self-canonicalising page for **any** string — the precise
defect #10453 removed, surviving in degraded mode.

Degraded pages now carry `noindex`. I first tried returning `503`, which
is arguably more correct, and backed it out: it broke eleven tests that
use the no-DB path as a rendering harness. `noindex` keeps the behaviour
those tests depend on and removes the indexing risk, which is the part
that matters. The path is unreachable in production — but "unreachable"
here means one misconfiguration away from indexable.

## 3. 404s were counted as successful page reads

`bot_fetch` ran as a router dependency. A dependency executes **before**
the handler and cannot see the response, so every miss was recorded as a
read.

It has moved to a middleware and gained a `status` property. Recording
the miss is right — an assistant asking for a URL that no longer exists
is how a library migration announces itself — but recording it as a page
view is a lie. Filter on `status` before reading anything else;
documented in `docs/reference/plausible.md`.

## Also

A docstring pointed at `app/src/router.tsx`, which does not exist.
Routing lives in `app/src/routes/index.tsx`.

## Verification

- `pytest tests/unit` — 1640 passed, including: degraded hub and impl
pages assert `noindex`, a companion test asserts normal pages do
**not**, and a `bot_fetch` test pins `status: "404"` on a miss
- `ruff check` + `ruff format --check` — clean
- `grep` confirms no remaining reference to the superseded policy in
`seo.md`, `nginx.conf` or `robots.txt`

## Not in this PR

The audit's other findings are handled elsewhere: the ten-day-red
monitor in #10478, and the analytics that recorded nothing in #10477.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants