Skip to content

ci: gate the CDN purge on the real Railway deployment status #1192

Description

@vivek7405

Problem

.github/workflows/purge-cdn.yml (added in #1189) purges the Cloudflare cache
after a site deploy, and gates the purge on a heuristic: it polls
GET /__webjs/version on the Railway origin until the reported uptime is
lower than the time elapsed since the run began, inferring "the process
restarted for THIS push".

That heuristic is wrong, and it failed on its very first live run. Run
30532402978 on merge commit 93883bbc failed at the wait step, and the purge
never ran. The cause: that commit touched only .github/workflows/purge-cdn.yml
and framework-dev.md, and Railway recorded the deployment as SKIPPED
(deployment 6603bade-6d78-4965-aeee-84050e7a1fa5). No deploy meant no restart,
so the wait loop ran its full 15 minute budget and then failed by design.

The refusal-to-purge logic behaved correctly (purging while the origin still
serves old bytes would repopulate the edge with them). The DEFECT is the
assumption that every push to main produces a Railway deploy. It does not, so
the workflow now red-Xes on every docs-only or CI-only commit. A CI job that
cries wolf on routine commits is worse than no job, because the real failures
stop being noticed.

Meanwhile the original staleness is still live: webjs-lockup-on-dark.svg and
webjs-lockup-on-light.svg are still the pre-#1185 masked versions at the edge,
so the mobile Safari logo bug still reaches visitors.

Design / approach

Stop inferring the deploy and ask Railway directly. Railway's public GraphQL
API exposes, per deployment, both a status and the originating commitHash,
which is precisely the signal needed. Confirmed against the live API (via the
Railway MCP list_deployments against this exact service):

6603bade | SKIPPED | 2026-07-30 09:52:06 UTC | 93883bbc  <- the failing case
5ae88e19 | SUCCESS | 2026-07-30 09:19:08 UTC | 086b3c75

So the wait step becomes: poll for the deployment whose commitHash equals
github.sha, then branch on its terminal status.

  • SUCCESS -> new bytes are live, purge.
  • SKIPPED -> Railway deliberately did not deploy this commit, so the origin
    is unchanged and there is nothing stale to evict. Exit 0 with a notice. This
    is the case that currently fails.
  • FAILED / CRASHED -> no new content reached the origin, so a purge would
    only cost a cold cache for no benefit. Exit 0 with a warning.
  • No deployment record at all within the budget -> exit 0 with a warning rather
    than failing, for the same reason.

This removes the uptime heuristic entirely, removes the false failures, and
makes the "did the content actually change" question exact instead of inferred.

workflow_dispatch keeps skipping the wait and purging immediately, which is
the manual escape hatch.

Implementation notes (for the implementing agent)

  • Where to edit: .github/workflows/purge-cdn.yml, the step named
    "Wait for the new build to be live on the origin". Replace its body. The
    "Purge the Cloudflare cache" and "Confirm the edge now matches the origin"
    steps are unchanged, but the purge step must become conditional on the wait
    step's outcome (use a step id plus an output such as
    should_purge=true|false, and gate the purge with
    if: steps.<id>.outputs.should_purge == 'true' || github.event_name == 'workflow_dispatch').
  • API details, all verified live:
    • Endpoint: https://backboard.railway.com/graphql/v2, HTTP POST, JSON body.

    • Auth: Authorization: Bearer <token> for an account token. A PROJECT token
      (the narrower, preferred CI credential) authenticates with the
      Project-Access-Token: <token> header INSTEAD of Authorization. The
      implementation should send the project-token header and fall back to
      Bearer, because which one applies depends on the token type the owner
      created, and this was not verifiable ahead of time (the local Railway CLI
      credential is not valid for the public API, it returns "Not Authorized").

    • Query shape (Relay pagination, edges { node { ... } }):

      query deployments($input: DeploymentListInput!, $first: Int) {
        deployments(input: $input, first: $first) {
          edges { node { id status createdAt meta } }
        }
      }

      with variables {"first": 20, "input": {"projectId": ..., "environmentId": ..., "serviceId": ...}}.
      The commit sha is on the node's meta object (meta.commitHash).

    • IDs for the website service (not secrets, they identify resources):

      • projectId 1d05ab75-b64e-4920-94ef-1ca7d04778fc
      • environmentId (production) 2e30b2f6-d166-48d3-9ae7-17176fb0942f
      • serviceId (@webjsdev/website) 2eb044b7-8698-43d3-959b-37c8925eb653
  • New secret required: RAILWAY_TOKEN. Document it in framework-dev.md
    beside the existing CLOUDFLARE_API_TOKEN note, including which token type
    and header. If the secret is absent, the step must fail with an explicit
    message rather than silently skipping the purge.
  • Landmines:
    • Railway's API sits behind Cloudflare and returns error code: 1010 to
      clients whose user agent looks automated (Python urllib is blocked;
      curl is fine). Use curl in the workflow.
    • A GraphQL error is returned with HTTP 200 and an errors array, so
      checking the HTTP status is NOT sufficient. Assert on .errors == null
      and on the parsed data, the same lesson as the --fail-with-body note
      already in this workflow for the Cloudflare call.
    • Deployment status is not immediately present: the record for a brand new
      push can take a few seconds to appear, so "no record yet" must keep
      polling rather than concluding SKIPPED.
    • Keep the existing hardening from ci: purge the Cloudflare cache after each site deploy #1189: || true on every jq extraction
      (a set -e abort on a transient non-JSON body was a real bug), the
      deadline-driven loop rather than an iteration count, and permissions: {}.
  • Invariants: none affected (CI only, no runtime or public API surface).
  • Tests + docs: no unit test layer applies to a workflow. Validate the YAML
    parses and every run block passes bash -n, and exercise the status
    branching against recorded API responses (a SUCCESS commit and the known
    SKIPPED commit 93883bbc) before merging. Update the framework-dev.md
    "CDN cache" section, which currently describes the uptime heuristic and will
    be stale.

Acceptance criteria

  • The wait step resolves the Railway deployment for github.sha and
    branches on its real status
  • A SKIPPED deploy exits 0 with a clear notice and does NOT purge, and does
    NOT fail the run (the ci: purge the Cloudflare cache after each site deploy #1189 regression)
  • A SUCCESS deploy purges
  • A missing RAILWAY_TOKEN fails loudly rather than skipping silently
  • workflow_dispatch still purges immediately with no wait
  • framework-dev.md updated: the uptime heuristic description replaced,
    and RAILWAY_TOKEN documented with its token type and header
  • Verified on a real merge that produces a deploy, and on one that Railway
    skips

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status
In progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions