Skip to content

fix(automation): run green unit baseline #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mkczarkowski merged 1 commit into from
Jul 27, 2026
Merged

fix(automation): run green unit baseline #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion context/map/artifact-1-territory.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- GENERATED by tools/repo-map/scan.mjs — do not hand-edit. Run `npm run repo-map`. -->

> Scan of `c823d6d` (2026-07-28) · window: last 12 months · engine: tools/repo-map
> Scan of `f67d3cc` (2026-07-28) · window: last 12 months · engine: tools/repo-map

**Where the work actually happens.** Git-history activity over the last 12 months, noise filtered (lockfiles, generated files, assets). Activity is a signal of *risk of accidental regression*, not of quality.

Expand Down
2 changes: 1 addition & 1 deletion context/map/artifact-2-structure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- GENERATED by tools/repo-map/scan.mjs — do not hand-edit. Run `npm run repo-map`. -->

> Scan of `c823d6d` (2026-07-28) · window: last 12 months · engine: tools/repo-map
> Scan of `f67d3cc` (2026-07-28) · window: last 12 months · engine: tools/repo-map

**How it is built.** Static import graph of `src` plus a file inventory of every module. Folders show where the code is; the graph shows what actually depends on what.

Expand Down
8 changes: 4 additions & 4 deletions context/map/artifact-3-contributors.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- GENERATED by tools/repo-map/scan.mjs — do not hand-edit. Run `npm run repo-map`. -->

> Scan of `c823d6d` (2026-07-28) · window: last 12 months · engine: tools/repo-map
> Scan of `f67d3cc` (2026-07-28) · window: last 12 months · engine: tools/repo-map

**Who to ask before you change something.** Bots and AI agents (Claude, Codex, Copilot, dependabot…) are filtered out — this is about humans who hold undocumented context. Answers "who has context on this area", not "who changed this line" (that is `git blame`).

Expand All @@ -12,16 +12,16 @@
| --- | --- |
| “mkczarkowski” | 77 |
| psmyrdek | 8 |
| Marcin Czarkowski | 4 |
| Marcin Czarkowski | 5 |
| Przemek Smyrdek | 2 |

## Who to ask, per active area

| Area | Ask (commits) |
| --- | --- |
| `src/lib` | “mkczarkowski” (29), psmyrdek (4) |
| `context/map` | Marcin Czarkowski (1) |
| `context/map` | Marcin Czarkowski (2) |
| `context/changes` | “mkczarkowski” (28) |
| `tests/e2e` | “mkczarkowski” (8), psmyrdek (3) |
| `src/commands` | “mkczarkowski” (19), psmyrdek (4) |
| `package.json` | “mkczarkowski” (7), Przemek Smyrdek (1) |
| `package.json` | “mkczarkowski” (7), Marcin Czarkowski (1) |
6 changes: 3 additions & 3 deletions context/map/repo-map.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- GENERATED by tools/repo-map/scan.mjs — do not hand-edit. Run `npm run repo-map`. -->

> Scan of `c823d6d` (2026-07-28) · window: last 12 months · engine: tools/repo-map
> Scan of `f67d3cc` (2026-07-28) · window: last 12 months · engine: tools/repo-map
>
> Operational map of the territory, built from cheap deterministic signals (git history + import graph), synthesized for decisions — not an essay. A new developer should know in ~15 min where things live, what is dangerous, and where to start. Refresh with `npm run repo-map`.

Expand Down Expand Up @@ -61,11 +61,11 @@ _No high-risk zones flagged this run._
| Area | Ask |
| --- | --- |
| `src/lib` | “mkczarkowski” (29), psmyrdek (4) |
| `context/map` | Marcin Czarkowski (1) |
| `context/map` | Marcin Czarkowski (2) |
| `context/changes` | “mkczarkowski” (28) |
| `tests/e2e` | “mkczarkowski” (8), psmyrdek (3) |
| `src/commands` | “mkczarkowski” (19), psmyrdek (4) |
| `package.json` | “mkczarkowski” (7), Przemek Smyrdek (1) |
| `package.json` | “mkczarkowski” (7), Marcin Czarkowski (1) |

## 6. First day — read these first

Expand Down
14 changes: 8 additions & 6 deletions context/night-loop.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,17 @@ outer gate.
after the turn; no repair or later iteration starts after the ceiling.
- One base commit is pinned for the entire run. If the remote base advances,
the loop stops instead of comparing against a stale baseline.
- Exact differential gate: lint cannot regress; passing tests cannot decrease;
the pinned set of 8 build-dependent failures must remain byte-for-byte the
same. Characterization work must increase the passing-test count.
- Exact differential gate: the root unit-test suite must stay green, lint
cannot regress, and the passing-test count cannot decrease. Build-dependent
smoke and E2E suites remain separate explicit gates outside this lint-only
loop.
- At most two repair turns in the same Codex session. Persistent failure opens
an `automation` issue and preserves the failed worktree for diagnosis.
- Successful iterations open one small PR each. The runner never merges PRs.
- Agent and gate environments omit credential variables. Codex and outer gates
run without network; binary diffs and patches matching common secret formats
are rejected.
- Agent and gate environments omit credential variables. Codex has no network
access; outer gates deny external network while allowing loopback-only test
fixtures. Binary diffs and patches matching common secret formats are
rejected.

The machine must remain awake and online. Saved Codex authentication and `gh`
authentication must already work.
Expand Down
31 changes: 24 additions & 7 deletions scripts/night-loop.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import {
mkdirSync,
openSync,
readFileSync,
readdirSync,
renameSync,
rmSync,
statSync,
Expand Down Expand Up @@ -38,7 +39,14 @@ const MAX_PATCH_BYTES = 1024 * 1024;
const DEFAULT_COMMAND_TIMEOUT_MS = 20 * 60 * 1000;
const DEFAULT_BASE = "claude/context-loops-bootstrap";
const VALID_REF = /^[A-Za-z0-9._/-]+$/;
const NO_NETWORK_PROFILE = "(version 1)(allow default)(deny network*)";
const NO_EXTERNAL_NETWORK_PROFILE = [
"(version 1)",
"(allow default)",
"(deny network*)",
'(allow network-bind (local ip "localhost:*"))',
'(allow network-inbound (local ip "localhost:*"))',
'(allow network-outbound (remote ip "localhost:*"))',
].join("");
const activeChildren = new Set();
let lockPath;
let fatalContext;
Expand All @@ -62,6 +70,15 @@ function sanitizedEnvironment() {
);
}

function unitTestArgs(worktree) {
const tests = readdirSync(join(worktree, "tests"), { withFileTypes: true })
.filter((entry) => entry.isFile() && entry.name.endsWith(".test.ts"))
.map((entry) => join("tests", entry.name))
.sort();
if (tests.length === 0) throw new Error("No root unit tests found");
return ["test", ...tests];
}

function parseArgs(argv) {
const options = {
deadline: "07:30",
Expand Down Expand Up @@ -466,7 +483,7 @@ async function captureBaseline({ repoRoot, base, deadlineMs, runRoot }) {
);
const lint = await runCommand({
command: "sandbox-exec",
args: ["-p", NO_NETWORK_PROFILE, "bun", "run", "lint"],
args: ["-p", NO_EXTERNAL_NETWORK_PROFILE, "bun", "run", "lint"],
cwd: baselineDir,
deadlineMs,
stdoutFile: join(logDir, "lint.log"),
Expand All @@ -476,7 +493,7 @@ async function captureBaseline({ repoRoot, base, deadlineMs, runRoot }) {
});
const tests = await runCommand({
command: "sandbox-exec",
args: ["-p", NO_NETWORK_PROFILE, "bun", "test"],
args: ["-p", NO_EXTERNAL_NETWORK_PROFILE, "bun", ...unitTestArgs(baselineDir)],
cwd: baselineDir,
deadlineMs,
stdoutFile: join(logDir, "test.log"),
Expand All @@ -492,8 +509,8 @@ async function captureBaseline({ repoRoot, base, deadlineMs, runRoot }) {
if (testSummary.fail !== failedTests.length) {
throw new Error("Baseline failing-test names do not match the failure count");
}
if (testSummary.fail !== 8) {
throw new Error(`Expected the pinned baseline of 8 failures, found ${testSummary.fail}`);
if (testSummary.fail !== 0) {
throw new Error(`Expected a green unit-test baseline, found ${testSummary.fail} failures`);
}
return { baseSha, lint: lintSummary, tests: testSummary, failedTests };
} finally {
Expand Down Expand Up @@ -606,7 +623,7 @@ async function validateIteration({
["diff", "git", ["diff", "--check"], true],
["typecheck", "bun", ["run", "typecheck"], true],
["lint", "bun", ["run", "lint"], true],
["test", "bun", ["test"], false],
["test", "bun", unitTestArgs(worktree), true],
["build", "bun", ["run", "build"], true],
["examples", "bun", ["run", "examples:catalog:check"], true],
["repo-map", "bun", ["run", "repo-map:check"], true],
Expand All @@ -616,7 +633,7 @@ async function validateIteration({
for (const [name, command, args, expectZero] of commands) {
const commandResult = await runCommand({
command: "sandbox-exec",
args: ["-p", NO_NETWORK_PROFILE, command, ...args],
args: ["-p", NO_EXTERNAL_NETWORK_PROFILE, command, ...args],
cwd: worktree,
deadlineMs,
stdoutFile: join(logDir, `gate-${name}.log`),
Expand Down