Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ What makes it worth wiring in:

- **Auto-fixes locally, fails in CI (same command).** Run it on your machine and it _fixes_ what it can (lint, formatting) instead of just complaining. Run it under CI and the identical command is check-only, so a PR can't merge with problems that should have been fixed.
- **Silent when green, so it's cheap to loop.** A passing run prints nothing and exits `0`, with no output to burn an agent's tokens or bury the one failure that matters. Agents can run it as often as they like.
- **Failure output written for an agent to act on.** When a check fails it names the tool it ran, the exact command, and a docs link, so the agent (or you) knows what to fix and how instead of guessing.
- **Failure output written for an agent to act on.** When a check fails it names the tool it ran, the exact argv, and a docs link, so the agent (or you) knows what to fix and how instead of guessing.
- **Convention over configuration.** Checks are just `verify:*` npm scripts run in parallel. Add, drop, or override any of them; there's no bespoke config format to learn.

## Install
Expand Down Expand Up @@ -127,11 +127,11 @@ Flags on the bare `verifyx` command:
| `circular-deps` | external | Circular dependencies ([skott](https://github.com/antoine-coulon/skott)). |
| `duplicate-code` | external | Copy-paste detection ([jscpd](https://github.com/kucherenko/jscpd)). |

External checks shell out to their tool and **skip gracefully when it is not installed**; `verifyx init` installs the ones you opt into. They run the tool from your local `node_modules/.bin` regardless of how `verifyx` was invoked. `oxlint`/`oxfmt`/`tsc` are resolved if present; the rest are declared as optional `peerDependencies`.
External checks spawn their tool and **skip gracefully when it is not installed**; `verifyx init` installs the ones you opt into. They run the tool from your local `node_modules/.bin` regardless of how `verifyx` was invoked. `oxlint`/`oxfmt`/`tsc` are resolved if present; the rest are declared as optional `peerDependencies`.

Because checks are named for their function, **on failure** an external check prints the tool it used, the exact command it ran, and a docs link, so you (or an agent) can add the tool's config (e.g. `knip.json`) without guessing. On success it prints nothing (output is buffered and flushed only on failure, to keep runs quiet and cheap). If you override a check with your own `verify:<name>` script, a failure shows that `npm run verify:<name>` was what ran.
Because checks are named for their function, **on failure** an external check prints the tool it used, the exact argv it ran, and a docs link, so you (or an agent) can add the tool's config (e.g. `knip.json`) without guessing. On success it prints nothing (output is buffered and flushed only on failure, to keep runs quiet and cheap). If you override a check with your own `verify:<name>` script, a failure shows that `npm run verify:<name>` was what ran.

**Passing extra arguments through.** When you run an external check directly, anything after `--` is forwarded verbatim to the underlying tool, so you can tweak an invocation without ejecting it: `verifyx circular-deps -- src/*.ts` runs skott against `src/*.ts`, `verifyx lint -- --quiet` passes `--quiet` to oxlint. `verifyx init` scaffolds `verify:circular-deps` as `verifyx circular-deps -- src/*.ts` (skott needs a target), so the default is visible in your `package.json` and easy to point at your own source layout.
**Passing extra arguments through.** When you run an external check directly, anything after `--` is forwarded as literal argv entries to the underlying tool, so you can tweak an invocation without ejecting it: `verifyx circular-deps -- src/index.ts` asks skott to start at that file, while `verifyx lint -- --quiet` passes `--quiet` to oxlint. With no entrypoint, skott scans the current project, so `verifyx init` scaffolds the cross-platform default `verifyx circular-deps` without a shell glob.

Each external check is configured through its **tool's own config file**, exactly as you would use that tool standalone:

Expand Down Expand Up @@ -326,7 +326,7 @@ import { analyzeComplexity, getCheck, runAll } from '@makerx/verify'
// Run the maintainability analysis directly.
const { failing } = analyzeComplexity({ pattern: 'src/**/*.ts', threshold: 50 })

// Run any single check by name, including the ones that shell out to an external tool.
// Run any single check by name, including the ones that spawn an external tool.
const lint = await getCheck('lint')?.runDefault()

const unused = await getCheck('unused-code')?.runDefault({ maxWarnings: 5 })
Expand Down
22 changes: 15 additions & 7 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,14 @@
},
"dependencies": {
"commander": "^15.0.0",
"cross-spawn": "^7.0.6",
"enquirer": "^2.4.1",
"minimatch": "^10.2.5",
"typescript": "^6.0.3"
},
"devDependencies": {
"@rollup/plugin-typescript": "^12.3.0",
"@types/cross-spawn": "^6.0.6",
"@types/node": "^26.1.1",
"@vitest/coverage-v8": "^4.1.10",
"better-npm-audit": "^3.11.0",
Expand Down
61 changes: 30 additions & 31 deletions src/checks/external.test.ts
Original file line number Diff line number Diff line change
@@ -1,94 +1,93 @@
import { describe, expect, it, vi } from 'vitest'

import { runCaptured } from '../shared/output.ts'
import { appendArgs, defineExternalCheck, externalFailureHint, runCountedBudget, selectCommand } from './external.ts'
import { defineExternalCheck, externalFailureHint, runCountedBudget, selectCommand } from './external.ts'

const fixable = { checkCommand: 'oxfmt --check .', fixCommand: 'oxfmt .' }
const notFixable = { checkCommand: 'tsc --noEmit' }
const fixable = { checkCommand: ['oxfmt', '--check', '.'], fixCommand: ['oxfmt', '.'] }
const notFixable = { checkCommand: ['tsc', '--noEmit'] }

describe('selectCommand', () => {
it('uses the fix command in fix mode when the check is fixable', () => {
expect(selectCommand(fixable, 'fix')).toBe('oxfmt .')
expect(selectCommand(fixable, 'fix')).toEqual(['oxfmt', '.'])
})

it('uses the check command in check mode', () => {
expect(selectCommand(fixable, 'check')).toBe('oxfmt --check .')
expect(selectCommand(fixable, 'check')).toEqual(['oxfmt', '--check', '.'])
})

it('always uses the check command when the check has no fix command', () => {
expect(selectCommand(notFixable, 'fix')).toBe('tsc --noEmit')
expect(selectCommand(notFixable, 'check')).toBe('tsc --noEmit')
expect(selectCommand(notFixable, 'fix')).toEqual(['tsc', '--noEmit'])
expect(selectCommand(notFixable, 'check')).toEqual(['tsc', '--noEmit'])
})
})

describe('externalFailureHint', () => {
it('names the tool, the exact command that ran, and the docs link', () => {
const hint = externalFailureHint({ name: 'unused-code', bin: 'knip', docs: 'https://knip.dev' }, 'knip --no-progress')
it('names the tool, the exact argv that ran, and the docs link', () => {
const hint = externalFailureHint({ name: 'unused-code', bin: 'knip', docs: 'https://knip.dev' }, ['knip', '--no-progress'])
expect(hint).toContain('unused-code')
expect(hint).toContain('knip')
expect(hint).toContain('knip --no-progress')
expect(hint).toContain('["knip","--no-progress"]')
expect(hint).toContain('https://knip.dev')
})

it('still names the tool and command when no docs link is set', () => {
const hint = externalFailureHint({ name: 'circular-deps', bin: 'skott' }, 'skott src')
const hint = externalFailureHint({ name: 'circular-deps', bin: 'skott' }, ['skott', 'src'])
expect(hint).toContain('skott')
expect(hint).toContain('skott src')
expect(hint).toContain('["skott","src"]')
expect(hint).not.toContain('undefined')
})
})

describe('appendArgs', () => {
it('appends passthrough args verbatim, unquoted (so shell globs still expand)', () => {
expect(appendArgs('skott --showCircularDependencies', ['src/*.ts'])).toBe('skott --showCircularDependencies src/*.ts')
})

it('returns the command unchanged when there are no extra args', () => {
expect(appendArgs('oxlint .', [])).toBe('oxlint .')
expect(appendArgs('oxlint .')).toBe('oxlint .')
})
})

describe('defineExternalCheck', () => {
it('exposes raw commands for eject, including the fix variant when fixable', () => {
const lint = defineExternalCheck({
name: 'lint',
description: '',
bin: 'oxlint',
checkCommand: 'oxlint .',
fixCommand: 'oxlint --fix .',
checkCommand: ['oxlint', '.'],
fixCommand: ['oxlint', '--fix', '.'],
devDeps: [],
})
expect(lint.eject).toEqual({ check: 'oxlint .', fix: 'oxlint --fix .' })
})

it('scaffolds default trailing args after `--` so a consumer can see and tweak them', () => {
it('scaffolds a bare CLI call and serializes argv only at the eject boundary', () => {
const circular = defineExternalCheck({
name: 'circular-deps',
description: '',
bin: 'skott',
checkCommand: 'skott',
checkCommand: ['skott'],
devDeps: [],
scaffoldArgs: 'src/*.ts',
})
expect(circular.scaffold.script).toBe('verifyx circular-deps -- src/*.ts')
expect(circular.scaffold.script).toBe('verifyx circular-deps')
expect(circular.eject).toEqual({ check: 'skott', fix: undefined })
})

it('scaffolds a bare CLI call when there are no default args', () => {
const knip = defineExternalCheck({ name: 'unused-code', description: '', bin: 'knip', checkCommand: 'knip', devDeps: [] })
const knip = defineExternalCheck({ name: 'unused-code', description: '', bin: 'knip', checkCommand: ['knip'], devDeps: [] })
expect(knip.scaffold.script).toBe('verifyx unused-code')
})

it('serializes eject lazily, so an unserializable command fails eject rather than check construction', () => {
const check = defineExternalCheck({ name: 'x', description: '', bin: 'tool', checkCommand: ['tool', '$HOME'], devDeps: [] })
expect(() => check.eject).toThrow('cannot safely serialize')
})
})

describe('runCountedBudget', () => {
const spec = { name: 'duplicate-code', bin: 'jscpd', checkCommand: 'jscpd src', docs: 'https://x' }
const spec = { name: 'duplicate-code', bin: 'jscpd', checkCommand: ['jscpd', 'src'], docs: 'https://x' }
const budget = (count: number, report = '') => ({ strategy: 'count' as const, unit: 'clone', count: async () => ({ count, report }) })

it('passes when the finding count is at or below the budget', async () => {
expect(await runCountedBudget(spec, budget(5), 5, [], {})).toEqual({ name: 'duplicate-code', ok: true })
})

it('passes one prebuilt argv, including passthrough args, to the counter', async () => {
const count = vi.fn(async () => ({ count: 0, report: '' }))
await runCountedBudget(spec, { strategy: 'count', unit: 'clone', count }, 5, ['--ignore', 'has space'], {})
expect(count).toHaveBeenCalledWith({ argv: ['jscpd', 'src', '--ignore', 'has space'], env: {} })
})

it('fails over budget, printing the counting run’s report instead of re-running the tool', async () => {
const transformingSpec = { ...spec, transformOutput: (output: string) => output.toUpperCase() }
const spy = vi.spyOn(console, 'error').mockImplementation(() => {})
Expand Down
Loading