Skip to content

Commit 4902cce

Browse files
Crash0v3rrid3claude
andcommitted
feat(DEVA11Y-657): automatic PR trigger, honest inputs, real outputs + severity gate
Addresses review feedback (inputs must be real, not for show; guide users to the automatic detection flow; VS Code output format) and the flag audit. action.yml: - Automatic flow (primary): trigger on pull_request opened/synchronize/reopened/ ready_for_review (actor = PR author, no commenter gate). Manual @mention stays as the secondary flow (with the commenter permission pre-check). - Trim inputs to the ones that do something: username, access-key, fail-on-severity. Removed comment / check-gate / comment-mode / inline-suggestions / sarif (now always-on server defaults) and remediation / ai-agent (deferred preview). check-gate is folded into fail-on-severity (none = never fail). - Real gate + outputs: parse the CLI's `[deva11y-result]` line into error-count / warning-count / findings-count / comment-url, and fail the run per fail-on-severity (a real `warning` gate, and `error` actually fails now — previously --pr-scan always exited 0 so the check never failed). Operational errors (FUP/permission/error) always fail. Dropped the sarif-file output (SARIF is uploaded server-side; there is no local file). README + example: - Automatic-first quick start (on: pull_request + issue_comment), @main (no v1 pin). - Inputs/outputs tables reflect reality; describe what actually gets posted (sticky comment, inline comments in VS Code format, Check Run, SARIF). - Inline are review comments, not one-click suggestion blocks (findings carry how-to-fix text, not literal replacement code) — corrected the earlier claim. - Single support link; removed the internal-note remediation section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent af30f19 commit 4902cce

3 files changed

Lines changed: 218 additions & 299 deletions

File tree

README.md

Lines changed: 46 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# BrowserStack Accessibility DevTools for GitHub
22

3-
Catch accessibility issues on your pull requests. Comment
4-
**`@AccessibilityDevTools`** on a PR and BrowserStack scans the changed code,
5-
then posts the findings right back on the PR — as a summary comment, inline
6-
suggestions, and an optional merge check.
3+
Catch accessibility issues on your pull requests. BrowserStack scans the changed
4+
code on every PR and posts the findings right back on the PR — a summary comment,
5+
inline comments on the offending lines, a merge check, and a SARIF upload to the
6+
Security tab.
77

88
- 🔎 **Scans only what changed** in the PR (fast, even on large repos).
9-
- 💬 **Results on the PR** — a summary comment and inline, one‑click suggestions.
10-
-**Optional merge gate** — fail the check when accessibility errors are found.
11-
- 🤖 **Optional agent hand‑off (preview)**the findings comment can `@mention` a PR
12-
agent you already use, which then acts under _your_ credentials.
9+
- 💬 **Results on the PR** — a sticky summary comment and inline comments.
10+
-**Merge gate** — fail the check when accessibility issues are found (configurable).
11+
- 🔐 **Zero write credentials on your runner**every write is posted by the
12+
BrowserStack Accessibility App; your workflow's `GITHUB_TOKEN` stays read-only.
1313

1414
Learn more: <https://www.browserstack.com/docs/accessibility-dev-tools/features/remediate-github>
1515

@@ -18,19 +18,24 @@ Learn more: <https://www.browserstack.com/docs/accessibility-dev-tools/features/
1818
## Prerequisites
1919

2020
1. **Install the BrowserStack Accessibility GitHub App** on your organization or
21-
repository (a onetime step, done by a repo/org admin).
21+
repository (a one-time step, done by a repo/org admin).
2222
2. **Add a BrowserStack Service Account key** as repository or organization
2323
**Actions secrets**:
2424
- `BROWSERSTACK_USERNAME`
2525
- `BROWSERSTACK_ACCESS_KEY`
2626

2727
## Quick start
2828

29-
Add `.github/workflows/browserstack-a11y.yml`:
29+
Add `.github/workflows/browserstack-a11y.yml`. It runs **automatically on every
30+
pull request**, and can also be re-run on demand by commenting
31+
`@AccessibilityDevTools` on the PR:
3032

3133
```yaml
3234
name: BrowserStack Accessibility DevTools
35+
3336
on:
37+
pull_request:
38+
types: [opened, synchronize, reopened, ready_for_review]
3439
issue_comment:
3540
types: [created]
3641

@@ -40,101 +45,69 @@ permissions:
4045
id-token: write # prove the run came from your CI (required)
4146

4247
concurrency:
43-
group: a11y-${{ github.event.issue.number }}
48+
group: a11y-${{ github.event.pull_request.number || github.event.issue.number }}
4449
cancel-in-progress: true
4550

4651
jobs:
4752
a11y:
48-
# Fires on a PR comment mentioning @AccessibilityDevTools. The action then verifies
49-
# the commenter has write/maintain/admin permission before scanning.
5053
if: >
51-
github.event.issue.pull_request &&
52-
contains(github.event.comment.body, '@AccessibilityDevTools')
54+
github.event_name == 'pull_request' ||
55+
(github.event.issue.pull_request &&
56+
contains(github.event.comment.body, '@AccessibilityDevTools'))
5357
runs-on: ubuntu-latest
5458
timeout-minutes: 15
5559
steps:
5660
- uses: browserstack/browserstack-accessibility-devtools-action@main
5761
with:
5862
username: ${{ secrets.BROWSERSTACK_USERNAME }}
5963
access-key: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
60-
comment: true
61-
check-gate: true
6264
fail-on-severity: error
63-
# inline-suggestions: true
64-
# sarif: true
65-
# comment-mode: update
66-
# remediation: true # preview; off by default
67-
# ai-agent: coderabbitai # bare name; the App posts "@coderabbitai"
68-
```
69-
70-
Then, on any pull request, comment:
71-
72-
```
73-
@AccessibilityDevTools
7465
```
7566
76-
The scan runs and results appear on the PR within a couple of minutes.
67+
That's it — open a PR and the results appear within a couple of minutes. To
68+
re-run manually, comment `@AccessibilityDevTools` on the PR.
7769

7870
## How it works
7971

80-
1. You comment **`@AccessibilityDevTools`** on the PR.
81-
2. The workflow runs BrowserStack's accessibility CLI against the PR's changed
82-
files, authenticated by your Service Account key.
83-
3. Results are posted back to the PR **by the BrowserStack Accessibility App**
84-
(a branded bot), so your workflow's default token never needs write access.
72+
1. On a pull request (or an `@AccessibilityDevTools` comment), the workflow runs
73+
BrowserStack's accessibility CLI against the PR's changed files, authenticated
74+
by your Service Account key.
75+
2. Results are posted back to the PR **by the BrowserStack Accessibility App** (a
76+
branded bot), so your workflow's default token never needs write access.
8577

86-
> **Why `id-token: write`?** GitHub mints a shortlived, reposcoped OpenID
78+
> **Why `id-token: write`?** GitHub mints a short-lived, repo-scoped OpenID
8779
> Connect token that proves the request genuinely came from this repository's CI
8880
> run. BrowserStack verifies it before posting. It carries **no** personal
8981
> identity and grants no standing access.
9082

83+
## What gets posted
84+
85+
On every scan the App posts, automatically:
86+
87+
- a **sticky summary comment** (one per PR, updated in place across runs);
88+
- **inline comments** on the offending lines, in the same format as the
89+
BrowserStack VS Code extension: `(rule): <description>` followed by how to fix it;
90+
- a **Check Run** with the pass/fail result;
91+
- a **SARIF upload** to GitHub code scanning (Security tab).
92+
9193
## Inputs
9294

93-
| Input | Default | Description |
94-
| -------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
95-
| `username` || **Required.** Service Account username (store as a secret). |
96-
| `access-key` || **Required.** Service Account access key (store as a secret). |
97-
| `comment` | `true` | Post a summary findings comment on the PR. |
98-
| `check-gate` | `true` | Publish a Check Run / commit status for the PR. |
99-
| `fail-on-severity` | `error` | Fail the check at this severity and above: `error`, `warning`, or `none`. |
100-
| `inline-suggestions` | `false` | Add one‑click suggestion blocks on offending lines. |
101-
| `sarif` | `false` | Publish results to GitHub code scanning (Security tab). Uploaded by the App server-side — your workflow token needs no extra permission. |
102-
| `comment-mode` | `update` | `update` a single sticky comment across runs, or post a `new` one each time. |
103-
| `remediation` | `false` | Enable the optional agent hand‑off (preview; see below). |
104-
| `ai-agent` || Required when `remediation: true`. The agent's name (e.g. `coderabbitai`, `claude`); we @‑mention it. |
95+
| Input | Default | Description |
96+
| ------------------ | ------- | ------------------------------------------------------------------------------------------------------- |
97+
| `username` | — | **Required.** Service Account username (store as a secret). |
98+
| `access-key` | — | **Required.** Service Account access key (store as a secret). |
99+
| `fail-on-severity` | `error` | Fail the check when findings at/above this severity exist: `error`, `warning`, or `none` (never fails). |
105100

106101
## Outputs
107102

108103
| Output | Description |
109104
| ---------------- | ------------------------------------ |
110105
| `result` | `pass` or `fail`. |
111-
| `error-count` | Number of errorseverity findings. |
112-
| `warning-count` | Number of warningseverity findings. |
106+
| `error-count` | Number of error-severity findings. |
107+
| `warning-count` | Number of warning-severity findings. |
113108
| `findings-count` | Total findings. |
114-
| `comment-url` | Link to the posted PR comment. |
115-
| `sarif-file` | Path to the SARIF file (if enabled). |
116-
117-
## Optional: agent hand‑off (preview)
118-
119-
Set `remediation: true` and name your agent with `ai-agent`. When findings are
120-
posted, the App `@mentions` that agent on the PR (for example, `@coderabbitai`), and
121-
**your** agent acts under **your** credentials and billing.
122-
123-
**Prerequisite:** if your agent ignores bot authors by default, allow‑list the
124-
**BrowserStack Accessibility** bot in its configuration.
125-
126-
```yaml
127-
with:
128-
username: ${{ secrets.BROWSERSTACK_USERNAME }}
129-
access-key: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
130-
remediation: true
131-
ai-agent: coderabbitai # BrowserStack posts "@coderabbitai"
132-
```
133-
134-
> This feature is a preview and is off by default. Hand‑off is supported only if
135-
> your AI agent accepts triggers from a bot/App comment; behaviour varies by agent,
136-
> and it is a silent no‑op if the agent isn't configured to accept it.
109+
| `comment-url` | Link to the posted PR summary comment. |
137110

138111
## Support
139112

140-
- <https://www.browserstack.com/support>
113+
<https://www.browserstack.com/support>

0 commit comments

Comments
 (0)