chore(skills): rewrite the rfc skill around this repo's RFC shape - #410
Open
ryanleecode wants to merge 3 commits into
Open
chore(skills): rewrite the rfc skill around this repo's RFC shape#410ryanleecode wants to merge 3 commits into
ryanleecode wants to merge 3 commits into
Conversation
The skill taught the Polkadot Fellowship template while every RFC in this repo follows docs/rfcs/0001-template.md; four files in docs/rfcs/ carry the upstream sections as a result. Its `context: fork` frontmatter also ran it as a background subagent, so the clarifying-question interview it specified could never reach the author, and it hardcoded an author name into the output shape. The body is now a routing layer over four references, and template.md is gone: it was a stale copy of the upstream template.
check-rfc.ts checks a draft's filename, frontmatter, H1 number, required sections, empty headings, surviving template text, unresolved markers, RFC 2119 keywords in descriptive sections, and its docs/rfcs/_index.md row. Every check is pinned by a fixture that makes it fire plus two valid RFCs, worded unlike each other, that must leave it silent. That matrix caught two checks that were wrong as first written: a parent heading whose subsections carry the content read as empty, as did a section holding only a code fence. Parsing comes from pinned @std modules rather than hand-rolled regex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
rfcskill now drafts RFCs in the shape this repo actually uses, and a draft can be checked before it is handed back. Three defects made the old version unable to do either.It taught the Polkadot Fellowship template —
Explanation,Stakeholders,Prior Art and References— while every RFC in this repo lives indocs/rfcs/and followsdocs/rfcs/0001-template.md(Summary,Motivation,Detailed Design,Drawbacks,Alternatives,Unresolved Questions). Four files indocs/rfcs/carry the upstream sections today; that drift came from this skill.Its frontmatter set
context: fork, which Claude Code runs as a background subagent by default. A background subagent cannot ask the author anything, so the clarifying-question interview the body spent 40 lines specifying could never run.It hardcoded
Authors | Valentin Sergeevinto the output shape, so every RFC it drafted would carry that name regardless of who wrote the proposal.Nothing in the old file mentioned
docs/rfcs/_index.md— the table reviewers use to find an RFC at all.What is new
A deterministic gate,
scripts/check-rfc.ts, run per draft:It checks filename,
title/ownerfrontmatter, H1 number against filename, required sections, headings with nothing under them, surviving template text,TODO/TBDmarkers, RFC 2119 keywords in descriptive sections, and the_index.mdrow.ERRORblocks handing a draft back;WARNwants a reason.The 139-line prose body became a 62-line routing layer plus four reference files, each answering one question: what to ask before drafting, what each section owes,
MUSTvsmust, and whether a draft is ready.template.mdis deleted — it was a stale copy of the upstream Fellowship template, anddocs/rfcs/0001-template.mdis the real one.Decisions a reviewer should weigh
Every rule is measured against the corpus, not asserted. Section counts come from the 16 numbered RFCs in
docs/rfcs/:SummaryandMotivation16/16,Drawbacks14/16,Alternatives12/16,Detailed Design10/16,Unresolved Questions8/16. Required-vs-expected in the gate follows those numbers.Two H1 forms pass.
RFC NNNN — Title(8 files) andRFC-NNNN: Title(6) are both accepted rather than picking one and declaring half the corpus wrong; the odd punctuation in0017warns.The gate is for the file you touched. Run with no paths it audits all 16 and reports 14 errors of existing drift — 6 files with no frontmatter, 4 using
## Explanation,0008's empty### Migration strategy,0024missing from_index.md. None of that is fixed here, and the skill says so explicitly so a future run does not "helpfully" rewrite the corpus.Pinned
@std/*over hand-rolled parsing. Frontmatter, arg parsing, globbing, and colour come fromjsr:@std/{front-matter,cli,fs,path,fmt};deno.lockis committed so the version that runs is recorded.Verification
deno test— 17 fixture tests pass. Each check is asserted to fire on a fixture that breaks it and to stay silent on two valid RFCs written in deliberately different styles.deno lintanddeno checkclean over both scripts.references/section-contract.mdexits 0 with no findings; the same draft with aTBDand no index row exits 1 namingMARKERandINDEX.Two checks were wrong when first written and the fixtures caught it before the gate was trusted: a parent heading whose subsections carry the content was reported empty, and a section whose only content is a code fence was too. Both were false positives across most of the corpus. Both now have silent-side tests, alongside
MARKERinside a fence andINDEXwith no index present.Not done here: the skill's own behavioural A/B and description-activation measurement, which need a sandboxed harness rather than this repo.
New concepts
Falsify a checker before you trust it. When you write a rule and the check that enforces it in the same sitting, both come out of the same mental phrasing — so the check passes on the example you had in mind and is silently wrong on everything else. The fix is a fixture matrix: for every check, a case that must make it fire, and at least two valid documents, worded unlike each other, that must leave it silent.
The silent half is the one that matters.
EMPTYlooked correct against a hand-written example and flagged## Detailed Designin nine of sixteen real RFCs, because a heading followed by### API changeshas no prose of its own:A test asserting
EMPTYfires would have passed in both versions. OnlyassertEquals(errors(GOOD_B), [])— a valid RFC that uses nested subsections — separates them.Use this wherever a check gates work: linters, validators, graders, CI rules. Skip it for a check whose failure is loud on its own, such as a type error or a build break.