feat(DEVA11Y-657): composite GitHub Action (v1 scan+report)#1
feat(DEVA11Y-657): composite GitHub Action (v1 scan+report)#1Crash0v3rrid3 wants to merge 8 commits into
Conversation
Scaffold the public browserstack-accessibility-devtools-action repo per
ADR-0035 Appendix A (GitHub App + Service Account model).
- action.yml: composite action. Inputs username/access-key (required, secret),
comment, check-gate, fail-on-severity, inline-suggestions, sarif, comment-mode,
remediation, ai-agent (validated ^[A-Za-z0-9-]+(\[bot\])?$). Outputs result,
error-count, warning-count, findings-count, comment-url, sarif-file. Hardcoded:
@AccessibilityDevTools trigger, diff scope, deva11y.config.* inference, commenter
write/maintain/admin permission gate. Steps implement fork-PR "diff, don't build"
safety (no install/postinstall of head code; skip when OIDC/secrets restricted),
pinned npx CLI (placeholder 1.3.0), Actions OIDC mint (aud placeholder
browserstack-accessibility-devtools), and POST to ci-review-service /v1/findings
with {platform,repo,prNumber,headSha,findings,scanProof,oidcToken} + X-CI-Platform
header. The Action holds no GitHub write credential; the App posts server-side.
- examples/browserstack-a11y.yml: reference consumer workflow (issue_comment,
read-only token + id-token: write, concurrency, if-guard).
- .github/workflows/release.yml: release stub (esbuild dist build guard, action.yml
validation, manual org-admin Marketplace publish note).
- README.md: public customer README (v1 = scan + report; agent hand-off preview
off by default; homepage browserstack.com).
- LICENSE (MIT), .gitignore.
Placeholders pending provisioning: CLI version 1.3.0, OIDC audience and service URL
(https://devtools-a11y-linter.browserstack.com), GitHub App. Draft PR.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…y token
The commenter permission pre-check called GET /collaborators/{user}/permission,
which requires push access. The documented least-privilege workflow token
(contents: read) cannot call it, so the request 403s, the `|| echo none`
fallback yielded "none", and the action refused to scan for EVERY commenter —
non-functional under its own recommended setup.
Make the client-side check best-effort: fast-refuse only on a DEFINITIVE
non-privileged role; on an inconclusive call (the read-only-token case) proceed
and rely on the authoritative server-side fail-closed permission gate in the
ci-review-service (ADR-0035 §5). The runner is the customer's and was never the
security boundary.
Also correct the preflight fork-safety comments: issue_comment workflows run in
base-repo context with secrets available even for fork PRs, so cred-absence is
not what protects fork PRs — the permission gate + "diff, don't build" are.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code reviewReviewed the composite action (bugs + security; no CLAUDE.md in this repo). Found 1 functional issue — now fixed in
Also corrected the preflight fork-safety comments: Verified clean: OIDC request URL separator ( No remaining high-confidence issues. Verdict: PASS (draft — placeholders for CLI version, service URL/audience, and the pinned action SHA remain, as expected pre-GA). 🤖 Generated with Claude Code |
- example workflow + README now reference @v1 (with a note to SHA-pin for production) instead of the @<commit-sha> placeholder - add .github/workflows/ci.yml: PR gate validating action.yml structure + YAML well-formedness (the repo previously had no PR checks) CLI version pin (action.yml) stays a documented release-time value — no pr-scan CLI version is published to npm yet; the release step pins the real version. GitHub App id / service URL remain provisioning placeholders. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-URL comments @browserstack/accessibility-devtools 1.5.0 is published (npmjs `latest`) and includes PR-scan mode, so pin CLI_VERSION 1.3.0 -> 1.5.0. The OIDC audience (browserstack-accessibility-devtools) already matches the deployed ci-review-service verifier config, and SERVICE_URL is the real global ingress; drop the stale PLACEHOLDER comments accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Title -> "BrowserStack Accessibility DevTools for GitHub"; workflow name -> "BrowserStack Accessibility DevTools". - Learn-more link -> /docs/accessibility-dev-tools/features/remediate-github. - Drop the internal-sounding "BrowserStack runs no AI model…what works today" copy from the intro bullet. - Remove the Service-Account "no user license…Enterprise plan" aside. - Inline the full ready-to-copy workflow in Quick start; drop the separate examples/ link. - Agent hand-off: replace the "what works today" internal note with a generic preview disclaimer, and add the bot allow-list as a prerequisite. - Remove Privacy & security and License sections. - Support -> single https://www.browserstack.com/support link. - examples/: use @main (always latest) instead of @v1; match workflow name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tion --head') The action called a nonexistent `scan` subcommand with --head/--changed-files/ --output and then minted OIDC + POSTed itself. Published CLI 1.5.0 does the whole PR-scan via `--pr-scan`: resolves the base..head diff locally (git), gets linter-server to normalize + HMAC-sign, mints the Actions OIDC token, and submits to the ci-review-service (POST <ci-service-url>/v1/findings) which posts as the App. - scan step -> `--pr-scan --include '**/*' --base <sha> --head-sha <sha> --repo <owner/name> --pr-number <n> --ci-service-url https://devtools-a11y-linter.browserstack.com` (--head-sha, not --head; drop --changed-files/--output/--format — the CLI derives the diff and submits itself). fail-on-severity=none -> --non-strict. - Remove the now-redundant `oidc` and `submit` steps (the CLI does both). - `report` gates on the CLI exit code (0 pass; 1/3/4/other fail). Known 1.5.0 limits (documented inline): --pr-scan does not forward the comment/ inline/sarif/remediation options or return per-severity counts / comment URL — the ci-review-service applies defaults; those outputs need a CLI enhancement. NOTE: --ci-service-url uses the shared host, so POST /v1/findings must be routed to ci-review-service on devtools-a11y-linter.browserstack.com (currently only /github is routed) for the submit to land. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
Also, the output message formatting should follow the same format as the VS Code formatting |
…+ severity gate Addresses review feedback (inputs must be real, not for show; guide users to the automatic detection flow; VS Code output format) and the flag audit. action.yml: - Automatic flow (primary): trigger on pull_request opened/synchronize/reopened/ ready_for_review (actor = PR author, no commenter gate). Manual @mention stays as the secondary flow (with the commenter permission pre-check). - Trim inputs to the ones that do something: username, access-key, fail-on-severity. Removed comment / check-gate / comment-mode / inline-suggestions / sarif (now always-on server defaults) and remediation / ai-agent (deferred preview). check-gate is folded into fail-on-severity (none = never fail). - Real gate + outputs: parse the CLI's `[deva11y-result]` line into error-count / warning-count / findings-count / comment-url, and fail the run per fail-on-severity (a real `warning` gate, and `error` actually fails now — previously --pr-scan always exited 0 so the check never failed). Operational errors (FUP/permission/error) always fail. Dropped the sarif-file output (SARIF is uploaded server-side; there is no local file). README + example: - Automatic-first quick start (on: pull_request + issue_comment), @main (no v1 pin). - Inputs/outputs tables reflect reality; describe what actually gets posted (sticky comment, inline comments in VS Code format, Check Run, SARIF). - Inline are review comments, not one-click suggestion blocks (findings carry how-to-fix text, not literal replacement code) — corrected the earlier claim. - Single support link; removed the internal-note remediation section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nt input Per review: keep remediation with a generic disclaimer (rather than dropping it). - action.yml: `ai-agent` input (optional, bare name); a validate-agent step (charset ^[A-Za-z0-9-]+(\[bot\])?$, fail fast); passes `--ai-agent` to the CLI only when set. Single input — presence enables the hand-off (no separate `remediation` boolean). - README: ai-agent row + a short "AI remediation hand-off (preview)" section with the generic disclaimer (supported if your agent accepts bot/App triggers). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks @aveek-bs — both of your points are addressed in the latest push:
While here, from a full audit of the inputs (your "are we supporting all of these?" note): the surface is now trimmed to the inputs that actually do something — |
DEVA11Y-657 — Composite GitHub Action (v1: scan + report)
Ships the public composite Action for the CI/CD accessibility flow (Epic DEVA11Y-463, ADR-0035/0036).
A developer comments
@AccessibilityDevToolson a PR → this Action runs the published@browserstack/accessibility-devtoolsCLI in PR-scan mode, authenticated by a BrowserStack Service Account key (zero GitHub credentials on the runner) → the CLI sends findings to linter-server, which normalizes + HMAC-signs them (scanProof) → the runner forwards{report, scanProof}(plus a best-effort Actions OIDC token) to the ci-review-service, which verifies and posts as the GitHub App.What's in this PR
action.yml— composite Action with the ADR-0035 Appendix A input surface (username,access-key,comment,check-gate,fail-on-severity,inline-suggestions,sarif,comment-mode,remediation,ai-agent) and outputs (result,error-count,warning-count,findings-count,comment-url,sarif-file).examples/browserstack-a11y.yml— reference consumer workflow (on: issue_comment,permissions: contents/pull-requests read + id-token write)..github/workflows/release.yml— versioned release/bundle pipeline (SHA-pinned CLI, taggeddist).README.md,LICENSE,.gitignore.Fork-PR safety
"Diff, don't build" — no
npm install/postinstallof untrusted head code; degrades to skip when OIDC/secrets are restricted.Status / dependencies
🤖 Generated with Claude Code