Fix detached test session output

[thebiglabasky]

Summary

• Make --detach for test, trigger, and pw-test return immediately after scheduling a recorded test session.
• Print the test session ID, test-sessions get <id> --watch, and the canonical app URL once, so agents and humans have actionable handles without waiting for result streaming.
• Reject --detach with file reporters (json, github) because detached runs exit before result reports can be written.
• Update the test-session skill guidance to prefer --detach for agent workflows and use test-sessions get --output json for machine-readable follow-up.
• Normalize frequency-like construct values before synthesizing check payloads, avoiding leaked Frequency objects when the config and CLI load separate module instances.

Validation

• pnpm --filter checkly exec vitest --run ./src/helpers/__tests__/test-helper.spec.ts ./src/services/__tests__/abstract-check-runner.spec.ts ./src/reporters/__tests__/abstract-list.spec.ts ./src/commands/__tests__/command-metadata.spec.ts
• pnpm --filter checkly exec vitest --run ./src/services/__tests__/abstract-check-runner.spec.ts ./src/reporters/__tests__/abstract-list.spec.ts ./src/constructs/__tests__/check.spec.ts ./src/commands/__tests__/command-metadata.spec.ts ./src/reporters/__tests__/json-builder.spec.ts ./src/reporters/__tests__/github-md-builder.spec.ts
• pnpm --filter checkly run prepare:dist
• pnpm --filter checkly run prepare:ai-context
• pnpm --filter checkly run prepack
• CHECKLY_SKIP_AUTH=1 packages/cli/bin/run skills investigate test-sessions
• CHECKLY_SKIP_AUTH=1 packages/cli/bin/run test --help
• CHECKLY_SKIP_AUTH=1 packages/cli/bin/run trigger --detach --reporter json exits with a clear unsupported-reporter error
• git diff --check

[thebiglabasky]

AI-RV agent review summary

@thebiglabasky I reviewed the detach scheduling path, reporter output changes, frequency-like value normalization, and the related unit coverage.

Verdict: approve from code-review standpoint, but GitHub will not allow this account to submit an approving review on its own PR. I did not find any defect-grade issues.

Verification:

• Focused Vitest passed after packaging the CLI fixture: abstract-check-runner, abstract-list, check, json-builder, and github-md-builder specs; 25 tests passed.
• git diff --check passed.
• Help-output smoke checks passed for test, trigger, and pw-test; the new --detach wording is present on all three commands.

[martzoukos]

This is a great AX upgrade 👌 .

One small bug that Claude surfaced in which the cli drops a couple of output options with detach:

---

--detach silently skips onEnd(), so --reporter json and --reporter github write no report file

What happens
In AbstractCheckRunner.run(), the new detach branch schedules the checks, emits RUN_STARTED + DETACH, and returns early — before this.emit(Events.RUN_FINISHED, …). RUN_FINISHED is the only event wired to reporter.onEnd() (test.ts:431, trigger.ts:199, pw-test.ts:387). So in detach mode onEnd() is never called for any reporter.

For two reporters, onEnd() is the only place output is produced:

• reporters/json.ts:78-96 — the writeFileSync of checkly-json-report.json lives entirely in onEnd(). In detach mode the file is never written.
• reporters/github.ts onEnd() — same pattern; the GitHub markdown summary file is never written.

(list.ts and ci.ts are mostly fine because onDetach() prints a human summary, though they skip the _printTestSessionsUrl() short link — the new onBegin reference lines cover the ID for them.)

Why this is a regression, not just a detach quirk
On main, a --detach run that ran to completion did emit RUN_FINISHED (abstract-check-runner.ts:146) and therefore did call onEnd() and write the json/github file. Only the Ctrl+C path skipped it. After this branch, detach always skips onEnd(). So anyone using --detach --reporter json (or github) to produce a report file gets nothing now — and there's no error or warning, the file just silently doesn't appear.

This is reinforced by the updated investigate-test-sessions.md, which steers agents toward both --detach and --reporter json — the combination an agent is most likely to use is exactly the broken one.

Note: this is a code-traced finding (I can't run it against a live backend in review), but the chain is structural: onEnd() is json/github's only write path, and onEnd() is unreachable under detach.

Repro

checkly test --detach --record --reporter json
# expected (old): ./checkly-json-report.json written
# actual (this branch): no file, no warning

Suggested fixes (pick one)

1. Preferred: still call the end-of-run path in detach. Either emit RUN_FINISHED after DETACH, or have the detach branch produce a report containing the testSessionId with an empty/scheduling checks array — so json/github emit a valid file an agent can parse for the session ID.
2. Make json/github onDetach() write their file (with the session ID and no results yet).
3. If "no file in detach" is acceptable, say so explicitly: emit a one-line warning when --detach is combined with a file reporter, and document it. Silent is the only outcome to avoid.

[thebiglabasky]

Why this is a regression, not just a detach quirk On main, a --detach run that ran to completion did emit RUN_FINISHED (abstract-check-runner.ts:146) and therefore did call onEnd() and write the json/github file. Only the Ctrl+C path skipped it. After this branch, detach always skips onEnd(). So anyone using --detach --reporter json (or github) to produce a report file gets nothing now — and there's no error or warning, the file just silently doesn't appear.

This is reinforced by the updated investigate-test-sessions.md, which steers agents toward both --detach and --reporter json — the combination an agent is most likely to use is exactly the broken one.

So that's a good catch 🙏🏻 (and I have an idea about why my agent did this 😬) but I differ on the conclusion.
By definition --detach won't finish anything, only kick things off. So while that's a good catch, I actually believe we shouldn't offer --reporter with it since there won't be results to report on unless we wait... which is exactly what a call without --detach does.
I'd fix the guidance and verify the exact behavior of reporters to ensure my hypothesis is correct though

[martzoukos]

Lfg then.