Skip to content

chore(skills): rewrite the rfc skill around this repo's RFC shape - #410

Open
ryanleecode wants to merge 3 commits into
mainfrom
rfc-skill-rework
Open

chore(skills): rewrite the rfc skill around this repo's RFC shape#410
ryanleecode wants to merge 3 commits into
mainfrom
rfc-skill-rework

Conversation

@ryanleecode

Copy link
Copy Markdown

The rfc skill now drafts RFCs in the shape this repo actually uses, and a draft can be checked before it is handed back. Three defects made the old version unable to do either.

It taught the Polkadot Fellowship template — Explanation, Stakeholders, Prior Art and References — while every RFC in this repo lives in docs/rfcs/ and follows docs/rfcs/0001-template.md (Summary, Motivation, Detailed Design, Drawbacks, Alternatives, Unresolved Questions). Four files in docs/rfcs/ carry the upstream sections today; that drift came from this skill.

Its frontmatter set context: fork, which Claude Code runs as a background subagent by default. A background subagent cannot ask the author anything, so the clarifying-question interview the body spent 40 lines specifying could never run.

It hardcoded Authors | Valentin Sergeev into the output shape, so every RFC it drafted would carry that name regardless of who wrote the proposal.

Nothing in the old file mentioned docs/rfcs/_index.md — the table reviewers use to find an RFC at all.

What is new

A deterministic gate, scripts/check-rfc.ts, run per draft:

deno run --allow-read=. -c .claude/skills/rfc/deno.json \
  .claude/skills/rfc/scripts/check-rfc.ts docs/rfcs/0027-your-rfc.md

It checks filename, title/owner frontmatter, H1 number against filename, required sections, headings with nothing under them, surviving template text, TODO/TBD markers, RFC 2119 keywords in descriptive sections, and the _index.md row. ERROR blocks handing a draft back; WARN wants a reason.

The 139-line prose body became a 62-line routing layer plus four reference files, each answering one question: what to ask before drafting, what each section owes, MUST vs must, and whether a draft is ready. template.md is deleted — it was a stale copy of the upstream Fellowship template, and docs/rfcs/0001-template.md is the real one.

Decisions a reviewer should weigh

Every rule is measured against the corpus, not asserted. Section counts come from the 16 numbered RFCs in docs/rfcs/: Summary and Motivation 16/16, Drawbacks 14/16, Alternatives 12/16, Detailed Design 10/16, Unresolved Questions 8/16. Required-vs-expected in the gate follows those numbers.

Two H1 forms pass. RFC NNNN — Title (8 files) and RFC-NNNN: Title (6) are both accepted rather than picking one and declaring half the corpus wrong; the odd punctuation in 0017 warns.

The gate is for the file you touched. Run with no paths it audits all 16 and reports 14 errors of existing drift — 6 files with no frontmatter, 4 using ## Explanation, 0008's empty ### Migration strategy, 0024 missing from _index.md. None of that is fixed here, and the skill says so explicitly so a future run does not "helpfully" rewrite the corpus.

Pinned @std/* over hand-rolled parsing. Frontmatter, arg parsing, globbing, and colour come from jsr:@std/{front-matter,cli,fs,path,fmt}; deno.lock is committed so the version that runs is recorded.

Verification

  • deno test — 17 fixture tests pass. Each check is asserted to fire on a fixture that breaks it and to stay silent on two valid RFCs written in deliberately different styles.
  • deno lint and deno check clean over both scripts.
  • End-to-end: a draft written by following references/section-contract.md exits 0 with no findings; the same draft with a TBD and no index row exits 1 naming MARKER and INDEX.

Two checks were wrong when first written and the fixtures caught it before the gate was trusted: a parent heading whose subsections carry the content was reported empty, and a section whose only content is a code fence was too. Both were false positives across most of the corpus. Both now have silent-side tests, alongside MARKER inside a fence and INDEX with no index present.

Not done here: the skill's own behavioural A/B and description-activation measurement, which need a sandboxed harness rather than this repo.

New concepts

Falsify a checker before you trust it. When you write a rule and the check that enforces it in the same sitting, both come out of the same mental phrasing — so the check passes on the example you had in mind and is silently wrong on everything else. The fix is a fixture matrix: for every check, a case that must make it fire, and at least two valid documents, worded unlike each other, that must leave it silent.

The silent half is the one that matters. EMPTY looked correct against a hand-written example and flagged ## Detailed Design in nine of sixteen real RFCs, because a heading followed by ### API changes has no prose of its own:

// wrong: any heading with no prose is empty
if (!section.content.trim()) err("EMPTY", );

// right: a parent that introduces subsections is not empty, and a
// section whose only content is a code fence is not empty either
if (section.content.trim() || section.hasChildren) continue;

A test asserting EMPTY fires would have passed in both versions. Only assertEquals(errors(GOOD_B), []) — a valid RFC that uses nested subsections — separates them.

Use this wherever a check gates work: linters, validators, graders, CI rules. Skip it for a check whose failure is loud on its own, such as a type error or a build break.

The skill taught the Polkadot Fellowship template while every RFC in this
repo follows docs/rfcs/0001-template.md; four files in docs/rfcs/ carry the
upstream sections as a result. Its `context: fork` frontmatter also ran it as
a background subagent, so the clarifying-question interview it specified
could never reach the author, and it hardcoded an author name into the
output shape.

The body is now a routing layer over four references, and template.md is
gone: it was a stale copy of the upstream template.
check-rfc.ts checks a draft's filename, frontmatter, H1 number, required
sections, empty headings, surviving template text, unresolved markers, RFC
2119 keywords in descriptive sections, and its docs/rfcs/_index.md row.

Every check is pinned by a fixture that makes it fire plus two valid RFCs,
worded unlike each other, that must leave it silent. That matrix caught two
checks that were wrong as first written: a parent heading whose subsections
carry the content read as empty, as did a section holding only a code fence.

Parsing comes from pinned @std modules rather than hand-rolled regex.
@ryanleecode
ryanleecode requested a review from a team August 14, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant