diff --git a/library/issues/backlog/issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md b/library/issues/backlog/issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md new file mode 100644 index 0000000..e558b3e --- /dev/null +++ b/library/issues/backlog/issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md @@ -0,0 +1,69 @@ + + +# IRD-022: /health never responds while indexing runs: brood prepare and registration resync execute synchronously on the event loop + +> **Status:** Backlog +> **GitHub issue:** [legioncodeinc/nectar#22](https://github.com/legioncodeinc/nectar/issues/22) (OPEN, filed 2026-07-04) +> **Severity:** P1 (liveness contract broken: the daemon is unreachable for minutes while indexing a large repo, so doctor and the Hive dashboard treat it as down) +> **Scope:** Event-loop starvation of `/health` DURING indexing. The post-index CPU busy-loop is a separate failure mode owned by [IRD-023](../issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md); the two share one investigation surface. + +--- + +## Problem + +`GET /health` on port 3854 times out indefinitely while a brood or registration cold resync runs on a large repository. The daemon is one Node process; `/health` is served by `node:http` on the same event loop that runs the indexing pipeline, and the pipeline contains multi-minute fully synchronous stretches during which the HTTP handler can never be scheduled. The socket accepts (kernel backlog) but the request is never serviced: exactly the reported Listen-but-timeout behavior. Fleet-status therefore omits nectar and the Hive dashboard stalls on "starting" forever. + +## Field report + +Environment: nectar 0.1.3 on Windows, honeycomb 0.2.3 fleet, ~85k-file repository. + +- Memory ramps to ~868 MB while the tree walk runs, plateaus, then declines to ~670 MB (walk finished). +- Port 3854 shows a clean Listen; earlier `/health` connections went CloseWait and then cleared. +- A fresh `GET /health` still times out indefinitely, even after indexing completes (the post-completion leg is IRD-023). +- Consequence: fleet-status omits nectar; the Hive dashboard never clears "starting". + +## Root cause (code-grounded, current main as of 2026-07-04) + +`/health` is served by `node:http` on the daemon's single event loop (`src/server.ts:76,108`). Three pipeline stages monopolize that loop: + +1. **Discovery blocks.** `spawnGitLsFiles` uses `spawnSync` (`src/brooding/discovery.ts:117-124`), and the walk fallback is a synchronous `readdirSync` generator (`src/registration/disk-fs.ts:100`). +2. **Content prep blocks hardest.** `prepareFiles` is a tight synchronous read+hash loop over every discovered file (`src/brooding/precheck.ts:126-133`); each `readContent` is a bare `readFileSync` (`src/registration/disk-fs.ts:70`) followed by hashing. On ~85k files this is minutes of uninterrupted synchronous work. +3. **The registration cold-catch-up resync is one macrotask.** `runCycle` (`src/registration/service.ts:290-320`) enumerates `fs.listPaths()` (sync walk) and runs `processOne` (statSync + readFileSync + hash + TLSH) for every path inside a while loop with no awaits and no yields. The function is `async` in name only on this path. + +While any of these run, no I/O callback (including the HTTP request handler) can fire. + +## Shipped mitigation (partial) + +PRD-019 dormant-by-default (PR #21) means a fresh boot no longer indexes anything, so `/health` answers immediately at boot and the Hive gate clears. This fixes the boot-time case only: the moment a project is activated and a brood or cold resync starts on a large repo, the starvation recurs. + +## Proposed remediation direction (proposed, not decided) + +- Move discovery + prepare + resync work off the event loop (worker thread), or make the loops yield (async iteration with per-N-files awaits and async fs), or both. +- Bound the per-macrotask work so no single synchronous stretch exceeds a small budget. +- Add a regression test asserting `/health` answers within a bounded time WHILE a large synthetic brood/resync is in flight. + +## Acceptance criteria + +| ID | Criterion | +|---|---| +| AC-1 | Given a large synthetic repo (tens of thousands of files) and an in-flight brood, when `GET /health` is issued at any point during discovery, prepare, or description, then it responds within a bounded time (target: under 250 ms) for the entire duration of the run. | +| AC-2 | Given a registration cold-catch-up resync over a large store, when `GET /health` is issued mid-resync, then it responds within the same bound; no single macrotask on the resync path exceeds the per-macrotask work budget. | +| AC-3 | Given the pipeline is restructured (worker thread and/or yielding loops), when the full brood completes, then its output (registered files, hashes, descriptions) is byte-equivalent to the pre-change synchronous pipeline. | +| AC-4 | A regression test exists in the suite that starts a large synthetic brood/resync and asserts `/health` latency stays within the bound while the work is in flight; the test fails against the pre-fix synchronous pipeline. | + +## Related + +- [IRD-023: post-index CPU busy-loop](../issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md) - the sibling failure mode from the same field report; it starves `/health` AFTER indexing. One investigation surface, two distinct fixes. +- [IRD-024: non-gitignore exclude for committed dirs](../issue-024-non-gitignore-exclude-for-committed-dirs/ird-issue-024-non-gitignore-exclude-for-committed-dirs.md) - independent, but reduces the size of the workload this IRD makes preemptible. +- [`prd-019-project-scoped-brooding-activation`](../../../requirements/backlog/prd-019-project-scoped-brooding-activation/prd-019-project-scoped-brooding-activation-index.md) - the shipped dormant-by-default mitigation covering the boot-time case only. +- `src/server.ts:76,108` - the `/health` handler on the shared event loop. +- `src/brooding/discovery.ts:117-124` - `spawnSync` discovery. +- `src/brooding/precheck.ts:126-133` - the synchronous `prepareFiles` loop. +- `src/registration/disk-fs.ts:70,100` - `readFileSync` content reads and the synchronous walk generator. +- `src/registration/service.ts:290-320` - the single-macrotask `runCycle` resync. diff --git a/library/issues/backlog/issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md b/library/issues/backlog/issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md new file mode 100644 index 0000000..77c7aa5 --- /dev/null +++ b/library/issues/backlog/issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md @@ -0,0 +1,58 @@ + + +# IRD-023: Post-index CPU busy-loop: per-batch known-path stat sweeps and per-event git check-ignore spawnSync keep the daemon hot after indexing + +> **Status:** Backlog +> **GitHub issue:** [legioncodeinc/nectar#23](https://github.com/legioncodeinc/nectar/issues/23) (OPEN, filed 2026-07-04) +> **Severity:** P1 (the daemon pegs a core indefinitely after indexing a large repo and `/health` stays unresponsive, so the process looks hung rather than idle) +> **Scope:** Sustained CPU burn and `/health` starvation AFTER the index walk finishes. Starvation DURING indexing is owned by [IRD-022](../issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md); the two share one investigation surface. + +--- + +## Problem + +After the index walk completes, the daemon does not settle into an idle steady state. CPU keeps climbing while memory stays flat or declines: the signature of a busy-loop, not of indexing. Every trickle of file-system events (editor autosave, build output, a log write) re-triggers work that scales with the size of the whole store rather than with the size of the event, and some of that work spawns a synchronous git process per path. `/health` stays unresponsive throughout. + +## Field report + +Environment: nectar 0.1.3 on Windows, honeycomb 0.2.3 fleet, ~85k-file repository. + +- Memory ramps to ~868 MB during the walk, plateaus, then declines to ~670 MB (walk finished). +- Past the 900-second mark, CPU keeps climbing with flat/declining memory: a busy-loop, not indexing. +- `/health` remains unresponsive the entire time, so fleet-status omits nectar and the Hive dashboard stalls on "starting". + +## Root cause (code-grounded, current main as of 2026-07-04): two named suspects + +1. **Per-batch known-path sweeps scale with store size, not batch size.** Every registration cycle batch recomputes `knownPaths()` and `missingPaths()` (`src/registration/service.ts:303-312`, per the snapshot comment), and `missingPaths` performs an `existsOnDisk` stat for EVERY known path. With ~85k rows in the store, a single settled watch event (one file save) triggers ~85k synchronous `statSync` calls. Any trickle of events re-runs the sweep, keeping the CPU pegged indefinitely while the event loop starves. This matches the flat-memory, climbing-CPU signature exactly. +2. **Shared-ignore cache misses spawn git synchronously per path.** `createSharedIgnore`'s gitignore leg resolves a snapshot miss via `runGitCheckIgnore` = `spawnSync("git", ["check-ignore", ...])` per path (`src/registration/ignore.ts:187-199`, wired at `src/registration/ignore.ts:282-291`). Any watch-event storm over paths not in the ls-files snapshot (untracked build artifacts, temp files) spawns a synchronous git process per observation; on Windows each spawn is tens to hundreds of milliseconds of blocked loop. + +Compounding: the recursive `fs.watch` intake on Windows can deliver high event volumes on large trees. Each settled event feeds suspect 1, and each raw observation can feed suspect 2 before debouncing, because the ignore test runs pre-debounce (`src/registration/fs-watch.ts:165-173`). + +## Proposed remediation direction (proposed, not decided) + +- Make the known/missing-path reconciliation incremental (indexes/dirty sets) instead of per-batch full sweeps: a single-path settle must be O(1)-ish, never O(store). +- Batch or cache negative check-ignore resolutions (and/or refresh the ls-files snapshot instead of per-path `spawnSync`); never spawn a process per watch event. +- Add a soak test: sustained single-file churn on a large synthetic store must keep CPU bounded and `/health` responsive. + +## Acceptance criteria + +| ID | Criterion | +|---|---| +| AC-1 | Given a store with tens of thousands of known rows, when a single watch event settles, then the reconciliation work performed is proportional to the event (O(1)-ish), not to the store; no full `knownPaths`/`missingPaths` stat sweep runs per batch. | +| AC-2 | Given a storm of watch events over paths absent from the ls-files snapshot, when the shared ignore predicate resolves them, then no synchronous process is spawned per event: negative resolutions are cached or the snapshot is refreshed in bulk. | +| AC-3 | Given sustained single-file churn (steady autosave-rate writes) on a large synthetic store, when the soak runs for a sustained window, then daemon CPU stays bounded (no monotonic climb) and `GET /health` remains responsive within a fixed latency bound throughout. | +| AC-4 | A soak/regression test encoding AC-3 exists in the suite and fails against the pre-fix per-batch-sweep behavior. | + +## Related + +- [IRD-022: /health starved during indexing](../issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md) - the sibling failure mode from the same field report; it starves `/health` DURING indexing. One investigation surface, two distinct fixes. +- [IRD-024: non-gitignore exclude for committed dirs](../issue-024-non-gitignore-exclude-for-committed-dirs/ird-issue-024-non-gitignore-exclude-for-committed-dirs.md) - independent, but shrinking the watched/known set reduces the constant factors here. +- `src/registration/service.ts:303-312` - the per-batch `knownPaths`/`missingPaths` sweep with an `existsOnDisk` stat per known row. +- `src/registration/ignore.ts:187-199,282-291` - `runGitCheckIgnore` via `spawnSync` on snapshot miss and its wiring. +- `src/registration/fs-watch.ts:165-173` - the pre-debounce ignore test that lets raw observations reach the spawn path. diff --git a/library/issues/backlog/issue-024-non-gitignore-exclude-for-committed-dirs/ird-issue-024-non-gitignore-exclude-for-committed-dirs.md b/library/issues/backlog/issue-024-non-gitignore-exclude-for-committed-dirs/ird-issue-024-non-gitignore-exclude-for-committed-dirs.md new file mode 100644 index 0000000..379d8ee --- /dev/null +++ b/library/issues/backlog/issue-024-non-gitignore-exclude-for-committed-dirs/ird-issue-024-non-gitignore-exclude-for-committed-dirs.md @@ -0,0 +1,59 @@ + + +# IRD-024: Document and surface a non-gitignore exclude for committed dirs (graph-ignore.json today, .nectarignore alias?) + +> **Status:** Backlog +> **GitHub issue:** [legioncodeinc/nectar#24](https://github.com/legioncodeinc/nectar/issues/24) (OPEN, filed 2026-07-04) +> **Severity:** P2 (capability exists and works; the gap is documentation and discoverability, but on large repos the cost of not finding it is a full walk over vendored code every brood) +> **Scope:** Documentation, discoverability, and dry-run surfacing of the existing non-gitignore exclude. Independent of the performance investigation in [IRD-022](../issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md) / [IRD-023](../issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md). + +--- + +## Problem + +Operators need to exclude directories from indexing that cannot be gitignored because they are committed (vendored code, checked-in build outputs). The capability already exists (`.honeycomb/graph-ignore.json`), but no user-facing documentation names it, so operators fall back to gitignore surgery that cannot cover committed paths, and there is no way to see which excludes are active before paying for a full walk. + +## Field report + +Environment: nectar 0.1.3 on Windows, honeycomb 0.2.3 fleet, ~85k-file repository (the same deployment whose memory/CPU timeline is recorded in IRD-022/IRD-023: ~868 MB peak during the walk, ~670 MB post-walk, sustained CPU climb afterwards). + +- The operator had to gitignore-exclude `.claude/worktrees` and `node_modules` to tame indexing. +- Committed vendored code (`vendor/`, 9,262 files) cannot be gitignored, so it is walked and described on every brood. +- Ask as filed: provide a non-gitignore exclude (`.nectarignore` or config) for committed dirs. + +## What already exists (code-grounded, and not discoverable) + +`.honeycomb/graph-ignore.json` at the repo root is exactly this: a per-repo JSON prefix list (`["vendor/"]` or `{ "ignore": ["vendor/"] }`) honored in UNION with gitignore semantics by the shared ignore contract (`src/registration/ignore.ts:43-127`). It is applied on the git discovery path AND the walk AND (since PRD-019d) every CLI discovery path. The reporter's exact case is a one-line file, no gitignore change needed. + +## Gaps to close + +1. **Documentation.** No user-facing doc names `graph-ignore.json` as the committed-dirs exclude. The README and first-run guidance should, with the `vendor/` example. +2. **Discoverability/naming.** Consider a `.nectarignore` file (gitignore syntax) as a friendlier alias feeding the same shared-ignore predicate, or at minimum surface the ignore sources and their effect in `nectar brood --dry-run` output ("N files excluded by graph-ignore.json"). +3. **Pre-brood visibility.** The dry-run report should make it obvious BEFORE a brood which excludes are active and how many files each source excludes, so an operator on a large repo can iterate without paying for a full walk. + +## Proposed remediation direction (proposed, not decided) + +- Author the user-facing documentation (README plus first-run guidance) naming `.honeycomb/graph-ignore.json` as the committed-dirs exclude, with the `vendor/` one-liner example. +- Decide on and, if accepted, implement the `.nectarignore` gitignore-syntax alias feeding the same predicate (union semantics preserved; no second ignore engine). +- Extend `nectar brood --dry-run` to report active ignore sources and per-source exclusion counts. + +## Acceptance criteria + +| ID | Criterion | +|---|---| +| AC-1 | Given a fresh reader of the README (or first-run output), when they need to exclude a committed directory, then the documentation names `.honeycomb/graph-ignore.json`, shows the `vendor/` example, and states the union-with-gitignore semantics. | +| AC-2 | Given a repo with an active `graph-ignore.json` (and `.nectarignore` if the alias ships), when `nectar brood --dry-run` runs, then the output lists each active ignore source and the count of files it excludes, without performing a full brood. | +| AC-3 | Given the `.nectarignore` alias is accepted and shipped, when both it and `graph-ignore.json` are present, then both feed the single shared-ignore predicate in union with gitignore semantics, and existing `graph-ignore.json` behavior is unchanged. | +| AC-4 | Given the reporter's scenario (committed `vendor/` tree), when the documented one-line exclude is added and a brood runs, then no file under `vendor/` is walked, prepared, or described, on the git discovery path, the walk fallback, and every CLI discovery site. | + +## Related + +- [IRD-022: /health starved during indexing](../issue-022-health-starved-by-synchronous-indexing/ird-issue-022-health-starved-by-synchronous-indexing.md) and [IRD-023: post-index CPU busy-loop](../issue-023-post-index-cpu-busy-loop/ird-issue-023-post-index-cpu-busy-loop.md) - the performance investigation from the same field report; this IRD is independent but shrinks the workload those fixes must bound. +- [`prd-019-project-scoped-brooding-activation`](../../../requirements/backlog/prd-019-project-scoped-brooding-activation/prd-019-project-scoped-brooding-activation-index.md) - PRD-019d hardened the ignore contract so `graph-ignore.json` applies on every CLI discovery path. +- `src/registration/ignore.ts:43-127` - the shared ignore contract implementing `graph-ignore.json` with union semantics. diff --git a/library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md b/library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md index 8f3eb25..3a1f542 100644 --- a/library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md +++ b/library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md @@ -19,13 +19,16 @@ The canonical resolution chain every product implements (mirrored, never importe ``` resolveFleetRoot(env, platform, home = os.homedir()): - 1. env.APIARY_HOME set and non-blank -> APIARY_HOME - (the installer's --home= pin is delivered as APIARY_HOME in the service environment) - 2. platform is linux AND env.XDG_STATE_HOME set and non-blank + 1. env.APIARY_HOME set, non-blank, and ABSOLUTE -> APIARY_HOME + (the installer's --home= pin is delivered as APIARY_HOME in the service environment; + the installer resolves a relative --home= against ITS cwd at capture time) + 2. platform is linux AND env.XDG_STATE_HOME set, non-blank, and ABSOLUTE -> join(XDG_STATE_HOME, "apiary") 3. otherwise -> join(home, ".apiary") ``` +Security amendment (2026-07-04, from the W3 security audits): env roots are honored ONLY when absolute; a relative value is ignored and the chain falls through. Honoring a relative value would anchor the fleet root, and everything derived from it (machine keys, trust roots, registry pidPaths), on `process.cwd()`, the exact footgun this ADR exists to prevent; the XDG Base Directory spec likewise requires ignoring relative values. Absoluteness is checked with `path.win32.isAbsolute` (a strict superset of the posix check) so a relative value is never mistaken for absolute on any host. + Per-product state is `resolveFleetRoot() + "/"`; the shared surface (registry.json, device.json, install-id) sits at the root itself. **3. The registry compatibility window contract.** One contract, stated once, binding all four repos during the migration window: @@ -35,6 +38,8 @@ Per-product state is `resolveFleetRoot() + "/"`; the shared surface (re - **Read side (doctor):** read new-first. When both files exist, the new file wins per daemon `name`; legacy-only entries merge additively. doctor never writes merged results back to the legacy file. - **Window close:** the legacy fallback (read side and write side) is removed only when every supported install path ships its migration. That removal is the close-out gate tracked in the superproject execution ledger (`library/ledger/EXECUTION_LEDGER-apiary-state-root.md`). +**4. Registry path strings: resolved absolute paths (confirmed 2026-07-04).** Entries written to the registry carry RESOLVED absolute `pidPath` / `telemetryDbPath` values (the writer's own `resolveFleetRoot` output), never tilde-literals. Rationale: a `~`-literal is expanded by the READER under the reader's home, which diverges from the writer's resolved root the moment `APIARY_HOME` or XDG overrides apply; nectar's entry builder already writes resolved absolute paths. Read-side, doctor continues to expand legacy tilde-literal entries during the compatibility window (existing behavior for old writers), and its telemetry trust coercion validates containment against its own resolved per-product trust roots plus the legacy root. + ## Context The Apiary is four separately-installable products that run as a coordinated local fleet: **doctor** (the supervisor + cross-daemon registry), **hive** (the always-on portal/dashboard), and **honeycomb** + **nectar** (the workloads). Every one of them roots its on-disk state at `~/.honeycomb`: diff --git a/library/qa/security/2026-07-04-security-audit.md b/library/qa/security/2026-07-04-security-audit.md new file mode 100644 index 0000000..b577a99 --- /dev/null +++ b/library/qa/security/2026-07-04-security-audit.md @@ -0,0 +1,38 @@ +# Security Audit - PRD-019 / PRD-020 (apiary root and activation) + +- **Date:** 2026-07-04 +- **Auditor:** security-worker-bee +- **Scope:** `feature/apiary-root-and-activation` @ `C:/Users/mario/GitHub/the-apiary/nectar` (uncommitted working tree, HEAD `d3b19e2`) carrying PRD-020 (apiary state-root migration) and PRD-019 (project-scoped brooding activation: 019a/019b/019d). +- **Ordering:** ran BEFORE `quality-worker-bee`, whose 2026-07-04 reports for PRD-019 and PRD-020 note these remediations as present in the diff under review. This record backfills the in-repo audit file the QA reports observed as missing (the pass itself ran and its fixes landed before QA). + +--- + +## Executive summary + +**One High finding, remediated in-session. One Low finding, remediated in-session. Three Low findings documented and accepted. No Critical findings. Gates green post-fix.** + +## Findings + +| # | Severity | Location | Finding | Disposition | +|---|---|---|---|---| +| H1 | **High** | `src/registration/gitignore.ts` | ReDoS: a crafted `.gitignore` in an untrusted, non-git bound repo could chain consecutive `**/` runs into overlapping unbounded RegExp quantifiers with exponential backtracking, freezing the single-threaded daemon on the git-absent walk path. | **Fixed**: `collapseGlobStars` collapses consecutive `**/` units (and `***`+) before compilation, bounding the compiled pattern while preserving gitignore semantics. | +| L1 | Low | `src/registration/brooding-state.ts` (and `src/state-migration.ts` state-dir creation) | The nectar state directory was created with default permissions. | **Fixed**: created `0o700` (owner-only) on first write. | +| L2 | Low | `/api/hive-graph/*` error bodies | Verbose error text in loopback API responses. | Documented, accepted: loopback-only, permission-gated surface. (The b-AC-6 persist-failure body was subsequently redacted in the 2026-07-04 QA remediation pass.) | +| L3 | Low | `src/hive-graph/active-projects.ts` | The pathological-root guard is lexical (path comparison), not an inode/realpath check; a symlinked alias of a guarded root could evade it. | Documented, accepted: defense in depth on top of explicit operator binding, not the primary control. | +| L4 | Low | temp-file names (`brooding-state.ts`, `state-migration.ts`, registry/projection writers) | Predictable temp names (`pid` + timestamp) for atomic writes. | Documented, accepted: files live in owner-only directories; rename is atomic. | + +## Checked clean + +- SQL guards: every identifier/value/LIKE routes through the `sql-guards.ts` helpers; no raw interpolation found in the new PRD-019/020 code. +- Loopback binding: the daemon still refuses to bind off-loopback with the default open permission gate (PRD-018j); the new `/projects` endpoints inherit the group gate. +- POST validation: the brooding-toggle body is hand-validated (closed value sets, non-empty `projectId`); the 1 MiB request-body cap covers the new endpoints. +- Token redaction: no credential or token value is logged or echoed by the new code paths. +- Containment: per-project contexts construct their disk fs rooted at the bound path (CWE-22 containment via `paths-safe.ts` unchanged); discovery for one project cannot enumerate another's tree. + +## Baseline verification + +| Check | After remediation | +|---|---| +| `npm run build` | clean | +| `npm run typecheck` | clean | +| Non-live suite | green (0 fail; live Deeplake network tests excluded per the orchestrator) | diff --git a/library/requirements/backlog/prd-019-project-scoped-brooding-activation/qa/2026-07-04-qa-report-prd-019c-hive-dashboard-surface.md b/library/requirements/backlog/prd-019-project-scoped-brooding-activation/qa/2026-07-04-qa-report-prd-019c-hive-dashboard-surface.md new file mode 100644 index 0000000..fea7fe1 --- /dev/null +++ b/library/requirements/backlog/prd-019-project-scoped-brooding-activation/qa/2026-07-04-qa-report-prd-019c-hive-dashboard-surface.md @@ -0,0 +1,73 @@ +# QA Report: PRD-019c Hive dashboard project activation and brooding toggle (hive-surface slice) + +**Plan document:** `nectar/library/requirements/backlog/prd-019-project-scoped-brooding-activation/prd-019c-hive-dashboard-project-activation.md` +**Audit date:** 2026-07-04 +**Base branch:** `main` (audit target is the uncommitted working tree of the hive repo) +**Head:** hive `feature/apiary-root-and-nectar-activation` (HEAD `de1350c`, all changes uncommitted) +**Auditor:** quality-worker-bee + +**Scope note:** this report covers ONLY the hive-repo surface PRD-019c specifies (the dashboard panel, the two `wire` methods, and their tests, implemented in `hive/src/dashboard/web/`). The nectar-side slices (019a active-project resolution, 019b endpoints and store, 019d ignore contract) are audited in a sibling report by the nectar-side QA agent; nothing in this file speaks to those. + +Ordering check: `security-worker-bee` ran first for this branch (report at `hive/library/qa/security/2026-07-04-security-audit.md`); its dashboard-relevant findings for this surface were XSS-clean, zod-fail-soft-clean, and one documented posture finding (F-4, CSRF on the pre-existing local-mode POST surface, explicitly deferred as a systemic follow-up). Correct order; no violation. + +## Summary + +Pass. All seven acceptance criteria (c-AC-1 through c-AC-7) are implemented with AC-named passing tests, the existing `FolderPicker` is consumed rather than forked, the two wire methods land on hive's aggregation client with the fail-soft unreachable posture the PRD requires, and the toggle placement follows the PRD's preferred option (the Hive Graph page, nectar's own page). Gates are green (`npm run typecheck` clean, `npm test` 384/384). No Critical findings, no Warnings; one Suggestion (pre-hydration control state). + +## Scorecard + +| Category | Status | Notes | +|---------------|--------|-------| +| Completeness | ✅ | c-AC-1..7 all traced to code with AC-named tests | +| Correctness | ✅ | Re-list-after-ack (never optimistic-only), unreachable disables controls, wire shapes match the 019b contract exactly | +| Alignment | ✅ | `FolderPicker` reused (not forked), placement on the Hive Graph page per the PRD's preferred option, BFF proxy path honored (hive ADR-0002) | +| Gaps | ✅ | Fail-soft wire, zod `.catch()` at every level; one minor pre-hydration edge (Suggestion) | +| Detrimental | ✅ | No regressions: page's pre-existing 015 tests still pass; shell wire mocks extended, not altered | + +## Critical Issues (must fix) + +None. + +## Warnings (should fix) + +None. + +## Suggestions (consider improving) + +- [ ] **Disable panel controls until first hydration**, `hive/src/dashboard/web/pages/hive-graph.tsx` (NectarProjectsPanel, `controlsDisabled = projectsWire.unreachable || busyKey !== null`) + + Before the first `nectarProjects()` poll resolves, the panel state is `EMPTY_NECTAR_PROJECTS` (`unreachable: false`, `globalBrooding: "on"`) with `hydrated === false`, so the global "Pause all brooding" button renders enabled while the body shows "loading...". An operator clicking in that window issues a write derived from the assumed default rather than nectar's actual state. Include `!hydrated` in `controlsDisabled`. Low impact (the window is one poll round-trip and the write itself round-trips through nectar's persisted truth), which is why this is a Suggestion, not a Warning. + +## Plan Item Traceability + +| # | Plan Requirement | Status | Implementation Location | Notes | +|--------|------------------|--------|--------------------------|-------| +| c-AC-1 | Zero active projects renders a "needs a project" empty state mounting the existing `FolderPicker`; no fabricated list | ✅ | `hive/src/dashboard/web/pages/hive-graph.tsx` (NectarProjectsPanel empty branch, `data-testid="nectar-needs-project"` + ``) | Test `c-AC-1` also asserts no project row renders | +| c-AC-2 | Bind ack `bound: true` triggers a re-list; the folder appears as an active brooding project after nectar's reconcile | ✅ | `hive-graph.tsx` (`onBound` -> `reList`); picker posts bind via the shared wire | Test `c-AC-2` drives the real `FolderPicker` DOM (select, name, bind) and asserts the row + `brooding` badge appear and `nectarProjects` was called at least twice | +| c-AC-3 | Active row shows the brooding state from `GET /api/hive-graph/projects` plus a toggle | ✅ | `hive-graph.tsx` (row render: badge via `broodingLabel`/`broodingBadgeTone` with exhaustive `never` checks; `data-testid="nectar-brooding-toggle"`) | Tests `c-AC-3` (page) and `c-AC-3` (wire, GET endpoint) | +| c-AC-4 | Toggle write reflects the new state after a re-list, driven by nectar's persisted truth, never optimistic-only | ✅ | `hive-graph.tsx` (`runBroodingWrite`: applies the ack read-model, then unconditionally `reList()`) | Tests `c-AC-4` (page: badge flips to `paused` from the re-listed state) and `c-AC-4` (wire: POST body `{ projectId, brooding }`) | +| c-AC-5 | Global pause shows every row `global-paused` and stops all brooding; clearing restores per-project states | ✅ | `hive-graph.tsx` (Panel `right` global button posting `{ global: "paused" | "on" }`) | Test `c-AC-5` covers pause and restore. The "nectar stops all brooding" half is nectar-side (019b b-AC-5), sibling report's scope | +| c-AC-6 | Nectar down: shell + surface render with nectar marked unreachable (PRD-015 AC-6); toggle disabled, never a throw | ✅ | `hive-graph.tsx` (`data-testid="nectar-projects-unreachable"` message; `controlsDisabled` includes `unreachable`; `runBroodingWrite` early-returns on unreachable) | Tests `c-AC-6` (page: global toggle `disabled === true`) and `c-AC-6` (wire: degrades to `UNREACHABLE_NECTAR_PROJECTS`); shell-level posture covered by the updated `shell-connectivity-gate` suite | +| c-AC-7 | Every value renders as escaped text; no token/secret in browse/bind/brooding bodies | ✅ | `hive-graph.tsx` (names/paths/labels as React text children only; no `dangerouslySetInnerHTML`); `wire.ts` (POST body is exactly `{ projectId, brooding }` or `{ global }` plus fixed session headers) | Test `c-AC-7` injects `` as name and path and asserts textContent-only rendering; security audit independently confirmed the XSS and secrets posture | +| N-1 | Wire addition: `nectarProjects(): Promise` -> `GET /api/hive-graph/projects`, fail-soft | ✅ | `hive/src/dashboard/web/wire.ts` (endpoint `hiveGraphProjects`; method with `NectarProjectsBodySchema.safeParse`, non-ok/parse-fail/throw all degrade to `UNREACHABLE_NECTAR_PROJECTS`) | Shapes (`globalBrooding`, `projects[{ projectId, name, path, brooding, watcher, counts }]`) match the 019b contract field-for-field | +| N-2 | Wire addition: `setNectarBrooding(body): Promise` -> `POST /api/hive-graph/projects/brooding`, fail-soft | ✅ | `wire.ts` (endpoint `hiveGraphBrooding`; ack parsed with the same schema; degrades to unreachable) | Body union matches 019b's two zod-validated forms exactly | +| N-3 | Reached through hive's BFF proxy, browser never talks to daemons directly (hive ADR-0002) | ✅ | Both endpoints are same-origin `/api/hive-graph/*` paths; hive's proxy owns the nectar prefix (pre-existing routing, unchanged) | | +| NG-1 | Do NOT fork `FolderPicker` or `ProjectsPage`; consume them | ✅ | `hive-graph.tsx` imports `FolderPicker` from `../folder-picker.js`; `folder-picker.tsx` is untouched in the diff; `projects.tsx` untouched | Honored | +| NG-2 | Placement: prefer the Hive Graph page; flag in review | ✅ | Panel mounted at the top of `HiveGraphPage`, above the existing needs-selection/graph content | Preferred option taken; flagged here per the PRD's instruction | + +## Files Changed (this slice) + +- `hive/src/dashboard/web/pages/hive-graph.tsx` (M), adds `NectarProjectsPanel` (poll, empty state + FolderPicker, per-project rows with badge + toggle, global pause) above the existing PRD-015 content +- `hive/src/dashboard/web/wire.ts` (M), adds the two nectar endpoints, the zod fail-soft wire schemas (`.catch()` at every level), `EMPTY_NECTAR_PROJECTS` / `UNREACHABLE_NECTAR_PROJECTS`, and the `nectarProjects` / `setNectarBrooding` client methods +- `hive/tests/dashboard/hive-graph-page.test.tsx` (M), adds the seven `c-AC-*` page tests (plus wire-mock plumbing) +- `hive/tests/dashboard/hive-graph-wire.test.ts` (M), adds `c-AC-3`/`c-AC-4`/`c-AC-6` wire tests for the two new methods +- `hive/tests/dashboard/shell-connectivity-gate.test.tsx` (M), extends the shell wire mock with the new methods so the PRD-015 AC-6 shell posture stays covered + +## Gate outputs + +- `npm run typecheck` (hive repo): clean (exit 0). +- `npm test` (hive repo): 56 files, 384 tests passed, 0 failed. + +## Verdict + +The hive-surface slice of PRD-019c passes cleanly: no Critical, no Warning, one opt-in Suggestion. End-to-end behavior against a live nectar (reconcile activating a bound folder, global pause actually stopping brooding) depends on the 019a/019b nectar slices and is the sibling nectar-side report's scope. diff --git a/library/requirements/backlog/prd-019-project-scoped-brooding-activation/qa/prd-019-project-scoped-brooding-activation-qa.md b/library/requirements/backlog/prd-019-project-scoped-brooding-activation/qa/prd-019-project-scoped-brooding-activation-qa.md new file mode 100644 index 0000000..2b9f4c6 --- /dev/null +++ b/library/requirements/backlog/prd-019-project-scoped-brooding-activation/qa/prd-019-project-scoped-brooding-activation-qa.md @@ -0,0 +1,147 @@ +# QA Report: PRD-019 Project-scoped brooding and activation control + +**Plan document:** `library/requirements/backlog/prd-019-project-scoped-brooding-activation/prd-019-project-scoped-brooding-activation-index.md` (plus sub-PRDs 019a / 019b / 019d; 019c is implemented in the hive repo and audited there, its nectar-side contract is checked below) +**Grounding:** the index's "Resolved decisions (confirmed 2026-07-04)" table and `library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md` +**Audit date:** 2026-07-04 +**Base branch:** `feature/apiary-root-and-activation` (HEAD `d3b19e2`; all implementation changes uncommitted in the working tree) +**Auditor:** quality-worker-bee + +Ordering note: `security-worker-bee` ran before this audit per the orchestrator. Its remediations are present in the diff under review: the ReDoS fix in the gitignore parser (`collapseGlobStars`, `src/registration/gitignore.ts:59-61`) and the owner-only `0o700` brooding-state directory mode (`src/registration/brooding-state.ts:173-176`). No 2026-07-04 security report file exists under nectar `library/qa/security/` (the audit appears to be recorded at the superproject level); noted as an observation, not an ordering violation. + +## Summary + +Pass with warnings. The daemon is genuinely dormant by default and multi-root: with zero bound projects it constructs no context, broods nothing, and `/health` reports `activeProjects: 0` with reason `no-active-projects`; the cwd fallback is gone from every daemon path (the remaining `process.cwd()` uses are CLI-verb-only and the unwired legacy `assembleDaemon` seam). All four in-repo AC groups (index AC-1..7, a-AC-1..7, b-AC-1..7, d-AC-1..7) trace to implementation, and the gates are green (build, typecheck, 706 non-live tests, 703 pass / 0 fail / 3 platform-conditional skips). Five Warnings, none an AC violation: the known primary-project-at-boot scoping of live enrichment and the hive-graph API (assessed independently below; the orchestrator's "no AC violated" read holds), a missing b-AC-4-named test, no `**`/ReDoS regression tests behind the security fix, the b-AC-6 "redacted error" returning raw filesystem error text, and dead `resolveBootProjection` code left by the multi-root rewire. + +## Scorecard + +| Category | Status | Notes | +|---------------|--------|-------| +| Completeness | ✅ | Every 019a/b/d AC implemented; index ACs composed and traced; 019c's nectar-side contract matches hive's consumer | +| Correctness | ✅ | Dormant-by-default, multi-root reconcile, pathological-root guard, ON defaults, and gitignore semantics (incl. negation/nested/anchored under the ReDoS fix) all verified | +| Alignment | ⚠️ | Live enrichment + hive-graph API scoped to the primary project at boot (documented follow-up); no AC violated, but the index Goal names the enrich leg | +| Gaps | ⚠️ | b-AC-4 and the `**`/ReDoS matcher behavior lack named tests; b-AC-7 tested at the parse/render layer only | +| Detrimental | ⚠️ | Dead `resolveBootProjection` in `src/cli.ts` (the PRD-011b boot pre-warm is no longer wired); raw error text in the b-AC-6 500 body | + +## Gate outputs + +- `npm run build` (tsc): pass. +- `npm run typecheck` (tsc --noEmit): pass. +- Non-live suite (`node --experimental-sqlite --test` over 62 files, excluding `hive-graph-search-live.test.ts`, `hive-graph-deeplake.test.ts`, `deeplake-transport.test.ts` per the orchestrator): 706 tests, 703 pass, 0 fail, 3 skipped (two POSIX-only permission tests skipped on win32, one symlink-permission skip). + +## Contract verification highlights + +- **No cwd fallback anywhere in the daemon.** `runDaemon` (`src/cli.ts:1084-1177`) never consults `process.cwd()` or `NECTAR_PROJECT_ROOT`; scope comes only from the bound active set. The legacy `projectRoot = options.projectRoot ?? process.cwd()` seam remains in `assembleDaemon` (`src/daemon.ts:545`) but is inert in the shipped daemon: without `registrationStore` / `asyncBroodStore` / `broodStore` wired, neither `triggerAutoBrood` (`src/daemon.ts:657-661`) nor the single-root registration pipeline (`src/daemon.ts:725-727`) can touch that root. Proven end-to-end by `test/daemon-active-projects.test.ts` (factory throws if ever invoked with zero active projects). +- **Multi-root reconcile.** `ProjectSupervisor.reconcile` serializes cycles, starts/stops/restarts-on-path-change per project (`src/project-supervisor.ts:85-123`), driven on the poll cadence plus on demand from the toggle API (`src/active-projects-runtime.ts:43-51`, `src/api/projects-api.ts:123-128`). +- **Pathological-root guard list.** `$HOME`, filesystem/drive roots, and `%WINDIR%\System32` are refused with `/health` reason `pathological-root` (`src/hive-graph/active-projects.ts:65-92,131-134`), matching the index's resolved-decision list exactly. +- **Brooding state at `~/.apiary/nectar/projects.json` with ON defaults.** `broodingStatePath` derives from `nectarStateDir` (`src/registration/brooding-state.ts:65-72`); `DEFAULT_PROJECT_BROODING = "on"`, `DEFAULT_GLOBAL_BROODING = "on"` (`:35-38`) per the resolved decisions; fail-soft loader, forward-compatible unknown-key warnings, atomic 0o700 first-write (`:99-180`). +- **Registry window write target** (shared surface with PRD-020): `defaultDoctorRegistryPath` writes `/registry.json` when the fleet root exists, else the legacy file, never both (`src/doctor-registry.ts:111-116`), per the ADR contract the index cites. +- **Gitignore parser semantics under the ReDoS fix.** `collapseGlobStars` collapses `**/` runs and `***`+ before compilation (`src/registration/gitignore.ts:59-61`); negation (`!`), nested `.gitignore` precedence, anchored patterns, dir-only patterns, and `.git/info/exclude` all pass their named tests (`test/gitignore.test.ts`), so the fix did not regress the specified semantics. The collapse preserves gitignore meaning by inspection (consecutive `**/` units are idempotent), but no test exercises a `**` pattern at all; see Warnings. +- **019c nectar-side contract match.** Hive's consumer (`hive/src/dashboard/web/wire.ts:943-981,2459-2486`) expects `GET /api/hive-graph/projects` returning `{ globalBrooding: "on"|"paused", projects: [{ projectId, name, path, brooding, watcher, counts }] }` and `POST /api/hive-graph/projects/brooding` with `{ projectId, brooding }` or `{ global }`, acking the same view. Nectar's `ProjectsView` (`src/projects-control.ts:29-47`) and `mountProjectsApi` (`src/api/projects-api.ts:94-131`) match field-for-field, including the honest `counts: null`. + +## Critical Issues (must fix) + +None. + +## Warnings (should fix) + +- [ ] **Live enrichment and the hive-graph API are scoped to the primary active project at boot**, `src/cli.ts:1146-1177,1216-1240` + + `primary` is computed once in `runDaemon`; the enricher loop and `mountHiveGraphApi` (search/build/status) are wired only when a primary active project exists at boot and stay bound to it. Consequences: (1) a daemon booted dormant that later gains a bound project runs brood + watch for it (the per-context path, including describe when Portkey is configured) but watch-appended pending version rows are not enriched until a restart, and the search/build endpoints stay unmounted until a restart; (2) with two active projects, only the first is enriched live. Independent assessment against the ACs: index AC-2 requires "begins brooding + watching it", which the context start path satisfies (`src/registration/project-context.ts:129-169`); a-AC-3/a-AC-4 speak of brood + watch contexts, which are per-project; b-AC-4's resume path runs brood + resync per context. No AC text is violated, confirming the orchestrator's read. It does drift from the index Goal "the daemon runs its watch + brood + enrich legs against the set of bound, brooding-enabled project directories" (index line 49), and the code marks it as a documented follow-up (`src/cli.ts:1146-1148`). Track the follow-up explicitly (ledger or a PRD) so the gap does not silently persist. + +- [ ] **b-AC-4 has no named test (OFF then ON resume with cold-catch-up resync)**, `test/projects-api.test.ts`, `test/project-supervisor.test.ts` + + The resume behavior is implemented (`createProjectContext.start()` runs auto-brood, hydrates, starts the watcher, and calls `service.requestResync()`, `src/registration/project-context.ts:129-169`; the supervisor restarts a newly-active project) and its pieces are covered by the a-AC-3/a-AC-7 context test and the a-AC-5 supervisor test, but no test drives the specific b-AC-4 sequence (project paused via the API, set back to `on`, reconcile resumes watch + brood + resync). The repo convention names each test after the AC it proves; b-AC-4 is the one AC in the four groups without a named test. + +- [ ] **No `**` semantics or ReDoS regression tests for the gitignore parser**, `src/registration/gitignore.ts:46-92`, `test/gitignore.test.ts` + + PRD-019d names `**` as a supported construct, and the security remediation (`collapseGlobStars`) specifically rewrote how `**` runs compile, yet `test/gitignore.test.ts` contains no pattern with `**` and no guard proving a crafted `.gitignore` (for example a long `**/**/**/...` chain against a deep path) completes quickly. The fix is correct by inspection, but the exact code path the security audit changed is the one path with zero test coverage. Add a `**` semantics case (for example `a/**/b` and `**/dist`) and a time-bounded pathological-pattern regression. + +- [ ] **b-AC-6's "redacted error" returns raw error text**, `src/api/projects-api.ts:117-121` + + On a persist failure the 500 body is `{ error: "persist_failed", reason: errorReason(err) }`, where `reason` is the raw `Error.message` from the filesystem (for example `EACCES: permission denied, open 'C:\Users\...\projects.json...'`), which leaks local paths to the caller. The endpoint is loopback-only and gated, so exposure is low, but b-AC-6 says "returns a redacted error". Return a stable machine code (the `error` field already is one) and drop or genericize `reason` for the persist branch. + + ```ts + } catch (err: unknown) { + return ctx.json({ error: "persist_failed", reason: errorReason(err) }, 500); + } + ``` + +- [ ] **Dead `resolveBootProjection` and the unwired PRD-011b boot pre-warm**, `src/cli.ts:962-998` + + The multi-root rewire stopped passing `bootProjection` into `assembleDaemon`, leaving `resolveBootProjection` defined but never called. Functionally the projection inherit still happens per project through the brood pipeline's disk load (`resolveProjection`, `src/brooding/pipeline.ts:183-188`), so no data path is lost when a brood runs; but the boot pre-warm that inherited hash-matched files WITHOUT brooding no longer runs in the shipped daemon, and the dead function will confuse the next reader. Either delete it or re-wire the pre-warm per active project. + +## Suggestions (consider improving) + +- [ ] **CLI usage text still describes the cwd default without scoping it to CLI verbs**, `src/cli.ts:132-133` + + "The project root comes from NECTAR_PROJECT_ROOT (defaults to the current working directory)" is now true only for the CLI verbs (`brood`, `prune`, `search` context, etc.), not the daemon. A one-line clarification ("the daemon's scope comes from bound projects; see `nectar projects`") would prevent operators from expecting `NECTAR_PROJECT_ROOT` to steer the daemon, which `runDaemon` no longer reads (the 019a implementation note had suggested keeping it as a documented power-user override; only the `options.projectRoot` test seam was kept). + +- [ ] **`nectar brooding on|off --all` issues one POST per project**, `src/cli.ts:938-946` + + Sequential per-project toggles are fine at dashboard scale, but each POST triggers a full reconcile. A batch body (or persisting all flags then reconciling once) would avoid N reconciles for N projects. + +## Plan Item Traceability + +Test names cite the AC they prove; all listed tests pass in the gate run above. + +| # | Plan Requirement | Status | Implementation Location | Notes | +|---|---|---|---|---| +| AC-1 | Empty/absent bindings: daemon broods nothing, watches nothing, `/health` reports zero with machine-readable reason, never a cwd brood | ✅ | `src/hive-graph/active-projects.ts:114-145`, `src/health.ts:131-136`, `src/cli.ts:1084-1177` | `test/daemon-active-projects.test.ts` "index AC-1 / a-AC-1 / a-AC-2"; `test/active-projects.test.ts` "index AC-1 / a-AC-1" | +| AC-2 | A dashboard bind activates the directory; nectar broods + watches it under its resolved tenancy id | ✅ | `src/projects-control.ts:65-83`, `src/registration/project-context.ts:86-169` | "index AC-2 / a-AC-3" test + daemon reconcile test; live enrichment scoping noted in Warnings (no AC text violated) | +| AC-3 | Two or more projects brood + watch independently; nothing outside the bound set discovered | ✅ | `src/project-supervisor.ts:90-123`, per-context fs rooted at the bound path (`src/registration/project-context.ts:90-92`) | "index AC-3 / a-AC-4" + supervisor tests; containment by per-root disk fs construction | +| AC-4 | Brooding OFF stops watch + brood, mints nothing new, leaves binding + Deeplake data untouched; ON resumes | ✅ | `src/api/projects-api.ts:106-130`, `src/project-supervisor.ts:96-107`, `src/registration/project-context.ts:170-188` | b-AC-3 test (persist-then-reconcile) + supervisor stop tests; `stop()` only stops/drains, touches no data; resume test gap tracked as the b-AC-4 Warning | +| AC-5 | Global pause overrides per-project state; `/health` reflects it; clearing restores | ✅ | `src/registration/brooding-state.ts:183-187`, `src/hive-graph/active-projects.ts:138-144,158` | b-AC-5 API test + "a global pause makes nothing active" resolution test | +| AC-6 | Non-git (or git-erroring) roots honor `.gitignore`; every CLI discovery path uses the shared predicate | ✅ | `src/registration/gitignore.ts`, `src/registration/ignore.ts:252-315`, `src/cli.ts:216-224,344-348,586-591,647,975-984,1044` | d-AC-1..7 named tests | +| AC-7 | Bind/unbind while running starts/stops within one reconcile cycle, no restart | ✅ | `src/active-projects-runtime.ts:43-51,60-65` | "a-AC-5 / index AC-7" daemon test | +| a-AC-1 | No bindings: no RegistrationService, no triggerAutoBrood, `/health` `activeProjects: 0` reason `no-active-projects` | ✅ | `src/daemon.ts:567-580,943-953`, `src/health.ts:131-136` | named tests (resolution + daemon) | +| a-AC-2 | cwd `$HOME`/`/`/`System32` with no bindings: nothing under cwd discovered or brooded | ✅ | `src/cli.ts:1084-1177` (no cwd consult), `src/daemon.ts:657-661` (no-op without stores) | daemon test's throwing factory proves no context is ever constructed | +| a-AC-3 | One bound project: exactly one context rooted at the bound path, scoped to `resolveProjectScope`'s id | ✅ | `src/cli.ts:1116-1131` (tenancy from creds + binding projectId), `src/registration/project-context.ts:86-98` | named tests (resolution + context) | +| a-AC-4 | Two projects: independent contexts, no cross-enumeration | ✅ | `src/project-supervisor.ts:90-123` | named tests | +| a-AC-5 | Bind/unbind mid-run: start/stop next reconcile, no restart | ✅ | `src/project-supervisor.ts:85-123` | named tests (supervisor + daemon) | +| a-AC-6 | Guarded roots refused, `/health` lists `refused: pathological-root` | ✅ | `src/hive-graph/active-projects.ts:65-92,131-134,168` | named tests (resolution + daemon) | +| a-AC-7 | Teardown stops watcher and drains bridge writes before release | ✅ | `src/registration/project-context.ts:170-188`, `src/active-projects-runtime.ts:74-81` | named tests (context + supervisor stopAll) | +| b-AC-1 | Missing file/dir reads as defaults (global on, project on); first write creates the dir | ✅ | `src/registration/brooding-state.ts:99-102,170-180` | named test | +| b-AC-2 | Malformed/non-object file warns and defaults, never throws | ✅ | `src/registration/brooding-state.ts:104-120` | named test | +| b-AC-3 | POST off persists, immediate reconcile stops watch + brood, data untouched | ✅ | `src/api/projects-api.ts:106-130`, `src/projects-control.ts:110-119` | named test (persist-before-reconcile ordering asserted) | +| b-AC-4 | Set back to on: reconcile resumes watch + brood with cold-catch-up resync | ⚠️ | `src/registration/project-context.ts:129-169` (`requestResync` on start) | Implemented; no b-AC-4-named test (Warning) | +| b-AC-5 | Global paused: nothing broods; `GET /projects` + `/health` report `global-paused`; on restores | ✅ | `src/registration/brooding-state.ts:183-187`, `src/hive-graph/active-projects.ts:158` | named test | +| b-AC-6 | Write failure: redacted error, prior state intact, no reconcile against a half-written file | ⚠️ | `src/api/projects-api.ts:115-122`, atomic write at `src/registration/brooding-state.ts:170-180` | Ordering + state preservation proven by the named test; `reason` carries raw error text (Warning) | +| b-AC-7 | CLI produces the same persisted + reconciled effect as the API; `nectar projects` reflects it | ⚠️ | `src/cli.ts:833-968`, thin client of the same endpoints (`src/api/loopback-client.ts:107-182`) | Satisfied by construction (same POST body, same handler); tests cover arg parsing + rendering only, no live-daemon round-trip | +| c-AC-1..7 | Hive dashboard empty state, toggle, wire methods, unreachable posture, escaping | 🟦 | hive repo (`hive/src/dashboard/web/wire.ts`, `pages/hive-graph.tsx`) | Audited separately in the hive repo; nectar-side contract verified to match field-for-field (see Contract verification highlights) | +| d-AC-1 | Non-git root: `dist/` + `*.log` excluded, `src/x.ts` included (walk fallback) | ✅ | `src/registration/gitignore.ts:182-189`, `src/registration/ignore.ts:302-311` | named tests (pure core + real temp-dir walk) | +| d-AC-2 | Nested `.gitignore` scoped to its own subtree | ✅ | `src/registration/gitignore.ts:222-265` | named test | +| d-AC-3 | Negation re-includes; other matches stay excluded | ✅ | `src/registration/gitignore.ts:98-130,168-174` | named test | +| d-AC-4 | Git present: `ls-files` snapshot authoritative, parser never consulted | ✅ | `src/registration/ignore.ts:302-311` | named test (fallback call counter asserted zero) | +| d-AC-5 | Git present-but-erroring stays loud, parser does not mask | ✅ | `src/registration/ignore.ts:303-309` (`lastError` branch) | named test | +| d-AC-6 | Every CLI discovery site passes the shared predicate | ✅ | `src/cli.ts:216-224,344-348,586-591,647,975-984,1044` | named test (non-git `discoverFiles` walk) + inspection of all five sites | +| d-AC-7 | `createDiskRegistrationFs` default is `createDefaultIgnore(root)`, not ignore-nothing | ✅ | `src/registration/disk-fs.ts:46-53` | named test | +| NG (index) | Shared `~/.deeplake/projects.json` schema unchanged; nectar reads, never writes it | ✅ | `src/projects-control.ts:70-73` (read via `loadProjectsCache` only) | No write path to the shared file exists in the diff | +| NG (index) | Service unit templates stay root-agnostic (no `NECTAR_PROJECT_ROOT` pin) | ✅ | `src/service/templates.ts` | Units carry only the optional `APIARY_HOME` state pin (PRD-020c), no project root | + +## Files Changed + +Implementation files relevant to PRD-019 (the same working tree also carries PRD-020; see that report): + +- `src/active-projects-runtime.ts` (A), the reconcile driver tying resolution to the supervisor and `/health` on the poll cadence +- `src/api/loopback-client.ts` (M), `projectsViaDaemon` / `setBroodingViaDaemon` loopback clients for the CLI +- `src/api/projects-api.ts` (A), `GET /projects` + `POST /projects/brooding` on the hive-graph group with hand-validated bodies +- `src/cli.ts` (M), multi-root `runDaemon` wiring (dormant-by-default, per-project contexts, primary-scoped enricher/API), `nectar projects` + `nectar brooding` verbs, shared ignore predicate on every CLI discovery site +- `src/daemon.ts` (M), the `activeProjects` seam, `reconcileActiveProjects()`, boot publish + reconcile loop arm, teardown drain +- `src/health.ts` (M), the `activeProjects` slice (count, reason, per-project brooding + watcher, refused list) +- `src/hive-graph/active-projects.ts` (A), pure active-set resolution, the pathological-root guard, the `/health` slice builder +- `src/index.ts` (M), export surface additions +- `src/project-supervisor.ts` (A), the serialized `Map` reconciler +- `src/projects-control.ts` (A), shared read-view + persist helpers used by both the API and the CLI +- `src/registration/brooding-state.ts` (A), the `~/.apiary/nectar/projects.json` fail-soft store with ON defaults and atomic 0o700 writes +- `src/registration/disk-fs.ts` (M), default predicate becomes `createDefaultIgnore(root)` (d-AC-7) +- `src/registration/gitignore.ts` (A), the dependency-free gitignore matcher (with the `collapseGlobStars` ReDoS guard) + disk loader +- `src/registration/ignore.ts` (M), the git-genuinely-absent parser fallback behind `eligible === null` (loud-error path preserved) +- `src/registration/project-context.ts` (A), the per-project brood + watch `RunningContext` with its own brood guard +- `test/active-projects.test.ts` (A), resolution + guard + health-slice tests (index AC-1/2/3, a-AC-1/3/4/6) +- `test/brooding-state.test.ts` (A), b-AC-1/2/6 + effective-state tests +- `test/daemon-active-projects.test.ts` (A), end-to-end daemon dormancy/reconcile/refusal tests (index AC-1/7, a-AC-1/2/5/6) +- `test/gitignore.test.ts` (A), d-AC-1..7 named tests +- `test/project-context.test.ts` (A), a-AC-3/a-AC-7 context lifecycle test +- `test/project-supervisor.test.ts` (A), a-AC-3/4/5/7 + serialization + path-change restart tests +- `test/projects-api.test.ts` (A), b-AC-3/5/6 + validation tests +- `test/projects-cli.test.ts` (A), b-AC-7 arg parsing + table rendering tests diff --git a/library/requirements/backlog/prd-020-apiary-state-root-migration/qa/prd-020-apiary-state-root-migration-qa.md b/library/requirements/backlog/prd-020-apiary-state-root-migration/qa/prd-020-apiary-state-root-migration-qa.md new file mode 100644 index 0000000..7f9e3a9 --- /dev/null +++ b/library/requirements/backlog/prd-020-apiary-state-root-migration/qa/prd-020-apiary-state-root-migration-qa.md @@ -0,0 +1,138 @@ +# QA Report: PRD-020 Apiary state-root migration + +**Plan document:** `library/requirements/backlog/prd-020-apiary-state-root-migration/prd-020-apiary-state-root-migration-index.md` (plus sub-PRDs 020a / 020b / 020c) +**Grounding:** `library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md` (Resolved decisions, including the absolute-only env-root amendment) +**Audit date:** 2026-07-04 +**Base branch:** `feature/apiary-root-and-activation` (HEAD `d3b19e2`; all implementation changes uncommitted in the working tree) +**Auditor:** quality-worker-bee + +Ordering note: `security-worker-bee` ran before this audit per the orchestrator. Its remediations are present in the diff under review: absolute-only env roots (`src/apiary-root.ts:33-35`, `win32.isAbsolute` per the ADR amendment, with a regression test in `test/apiary-root.test.ts`), and the owner-only `0o700` state-dir mode (`src/state-migration.ts:87,125`, `src/registration/brooding-state.ts:176`). No 2026-07-04 security report file exists under nectar `library/qa/security/` (the audit appears to be recorded at the superproject level); noted as an observation, not an ordering violation. + +## Summary + +Pass with warnings. All four AC groups (index AC-1..8, a-AC-1..6, b-AC-1..7, c-AC-1..6) trace to implementation, and every automatable AC has a named passing test; `npm run build`, `npm run typecheck`, and the non-live suite (62 files, 706 tests, 703 pass / 0 fail / 3 platform-conditional skips) are green. Three Warnings: a present-but-malformed registry file aborts daemon boot inside the migration pass (harsher than the fail-soft boot posture 020b specifies), the registry refresh can briefly advertise a `telemetryDbPath` that does not exist yet when the SQLite move failed, and c-AC-6 (live doctor probe during the coordination window) remains unverified in-repo as the AC itself anticipates. None blocks ship; the first Warning is the one worth fixing before the branch merges. + +## Scorecard + +| Category | Status | Notes | +|---------------|--------|-------| +| Completeness | ✅ | Every index / a / b / c AC implemented; all automatable ACs have AC-named passing tests | +| Correctness | ⚠️ | Two edge-case deviations from 020b/020c posture (boot abort on malformed registry; transient stale `telemetryDbPath` after a failed SQLite move) | +| Alignment | ✅ | ADR-0005 chain implemented verbatim, including the absolute-only amendment; non-moving surfaces untouched | +| Gaps | ⚠️ | c-AC-6 deferred to manual/window verification; no explicit AC-8-named no-change assertion (covered by existing suites + inspection) | +| Detrimental | ✅ | No data-integrity, secret, or build issues found | + +## Gate outputs + +- `npm run build` (tsc): pass. +- `npm run typecheck` (tsc --noEmit): pass. +- Non-live suite (`node --experimental-sqlite --test` over 62 files, excluding `hive-graph-search-live.test.ts`, `hive-graph-deeplake.test.ts`, `deeplake-transport.test.ts` per the orchestrator): 706 tests, 703 pass, 0 fail, 3 skipped (two POSIX-only permission tests skipped on win32, one symlink-permission skip). + +## Critical Issues (must fix) + +None. + +## Warnings (should fix) + +- [ ] **Malformed registry file aborts daemon boot inside the migration pass**, `src/state-migration.ts:167-178` + `src/daemon.ts:839-856` + + `runStateMigration` performs the c-AC-4 registry refresh via `registerWithDoctor`, which throws `DoctorRegistryError` on a present-but-malformed registry file. The daemon's `start()` calls `runStateMigration` with no try/catch, so a corrupt shared registry (a file any fleet product's installer writes) permanently blocks nectar's boot and prevents the migration marker from ever being written, even though every file move already succeeded. PRD-020b specifies the pass as fail-soft at the file level ("the daemon still boots"); c-AC-4's "fails loudly without being clobbered" is satisfied on the not-clobbering half, but bricking boot goes beyond loud. `test/state-migration.test.ts:225-247` proves the throw and non-clobber; nothing proves the daemon survives it. Recommended: catch the refresh failure in the boot path, log it loudly, boot anyway, and retry on the next boot. + + ```ts + // src/state-migration.ts:169 (throws DoctorRegistryError upward) + registerWithDoctor({ config: {...}, registryPath }); + // src/daemon.ts:843 (uncaught in start()) + runStateMigration({ config: {...}, log }); + ``` + +- [ ] **Registry refresh can advertise a `telemetryDbPath` that does not exist yet**, `src/state-migration.ts:165-178` + + `shouldRefreshRegistry` is true whenever any legacy state exists, including when the telemetry SQLite move FAILED (`failed.length > 0`). The refreshed entry derives `telemetryDbPath` from the new runtime dir (`src/doctor-registry.ts:134-146`), so doctor is pointed at a path that will not exist until the next boot's retry succeeds, while the daemon itself falls back to the legacy DB via `resolveStateReadPath` (`src/daemon.ts:851-855`). This contradicts the 020c implementation note "the registry refresh must run AFTER the telemetry SQLite has moved ... so `telemetryDbPath` never points at a path that does not exist yet". Transient (heals on the retry boot), but doctor would report a missing DB during the window. Recommended: skip or defer the refresh for the entry's DB pointer when `telemetry/nectar.sqlite` is in `failed`. + +- [ ] **c-AC-6 unverified in-repo (doctor probe end-to-end during the window)**, `prd-020c-service-unit-and-doctor-registry-adoption.md:54` + + The AC itself calls for "manual or integration verification against a real doctor build during the window"; no evidence of that verification exists in this repo (no integration test, no recorded manual run). The write-side contract (c-AC-3) and refresh mechanics (c-AC-4) are tested, so the risk is contained, but the AC should not be marked done until the cross-repo probe is exercised and recorded (the superproject execution ledger is the natural home). + +## Suggestions (consider improving) + +- [ ] **Stale `~/.honeycomb` doc comments on moved surfaces**, `src/lock.ts:18`, `src/api/daemon-api-wiring.ts:86`, `src/hive-graph/search.ts:42`, `src/hive-graph/search-types.ts:59`, `src/enricher/config.ts:4,35` + + 020a's implementation note asked for the code to stop asserting the superseded shared-dir design; `config.ts` and `config-file.ts` were updated, but these comments still describe `~/.honeycomb/nectar.pid` / `~/.honeycomb/nectar.json` for state that now lives under `~/.apiary/nectar/`. + +- [ ] **Service state-dir derivation ignores `NECTAR_RUNTIME_DIR`**, `src/service/index.ts:42-44`, `src/service/templates.ts:31-34` + + `serviceStateDir` / `launchdLogDir` honor only `plan.apiaryHome`, hardcoding `/.apiary/nectar` otherwise. 020c's solution text says the plan should honor `APIARY_HOME` / `NECTAR_RUNTIME_DIR` at install time the same way the daemon does at runtime. No AC requires the `NECTAR_RUNTIME_DIR` leg (c-AC-1/2 test only the default and `APIARY_HOME`), so this is a nice-to-have alignment with the narrative, not a gap. + +- [ ] **Failed-SQLite fallback makes the telemetry DB a legacy WRITE target**, `src/daemon.ts:851-855` + + When the SQLite migration fails, `resolveStateReadPath` hands the daemon the legacy DB path and the daemon then writes to it, while 020b states "Writes NEVER target the legacy location after 020a lands. The fallback is read-only." Continuity over purity is a defensible tradeoff here (better than forking telemetry history), but it deserves either a comment acknowledging the exception or a follow-up to converge with the stated rule. + +- [ ] **No explicit AC-8-named no-change assertions**, `prd-020a-apiary-root-helper-and-path-adoption.md:60` + + 020a asked for the non-moving literals (`DEFAULT_PROJECTION_REL_PATH`, `GRAPH_IGNORE_FILE`, `ALWAYS_IGNORED_SEGMENTS`, the `~/.deeplake` surface) to be listed as explicit no-change assertions. They are unchanged (verified by inspection: `src/projection/format.ts:13`, `src/registration/ignore.ts:44,47`) and pinned indirectly by existing suites, but no test names AC-8. + +## Plan Item Traceability + +Test names cite the AC they prove; all listed tests pass in the gate run above. + +| # | Plan Requirement | Status | Implementation Location | Notes | +|---|---|---|---|---| +| AC-1 | Fresh install creates all nectar state under `~/.apiary/nectar/`, nothing nectar-owned in `~/.honeycomb` | ✅ | `src/apiary-root.ts:44-74`, `src/config.ts:112-115`, `src/state-migration.ts:125` | Composed from a-AC/b-AC tests; the registry-entry exception now follows the ADR window contract (new file when the fleet root exists), which supersedes AC-1's "sole exception" wording per c-AC-3 | +| AC-2 | `APIARY_HOME` drives the root; `NECTAR_RUNTIME_DIR` still wins for nectar's state dir | ✅ | `src/config.ts:112-115` | `test/config.test.ts` "a-AC-2 resolveConfig default runtimeDir follows APIARY_HOME/nectar"; `test/apiary-root.test.ts` "a-AC-4" | +| AC-3 | Upgrade migrates the four files; second boot idempotent; no failed-migrate deletion | ✅ | `src/state-migration.ts:118-183` | `test/state-migration.test.ts` b-AC-1 / b-AC-2 / b-AC-3 | +| AC-4 | Crash-safe: no state loss, legacy pid still guards, retry completes | ✅ | `src/state-migration.ts:48-58,85-96` | b-AC-3 / b-AC-4 / b-AC-7 tests | +| AC-5 | Readers fall back to legacy when new absent; post-migration writes land new-only | ✅ | `src/state-migration.ts:28-36`, `src/config-file.ts:96-106`, `src/telemetry-usage/emit.ts:206-248` | b-AC-5 + fixes.test.ts a-AC-5 pair + telemetry-usage a-AC-5 pair; see Suggestion on the failed-SQLite write nuance | +| AC-6 | launchd log dir under the resolved state dir; no `.honeycomb` literal in templates; index.ts creates exactly that dir | ✅ | `src/service/templates.ts:31-34`, `src/service/index.ts:228-233` | `test/service-templates.test.ts` "c-AC-1"; `test/service-index.test.ts` AC-018l.9 (updated) + c-AC-2 | +| AC-7 | Registry entry values carry new paths; file target per window contract | ✅ | `src/doctor-registry.ts:111-116,134-146`, `src/state-migration.ts:165-178` | b-AC-1 registry assertions + doctor-registry c-AC-3 pair | +| AC-8 | Non-moving surfaces byte-identical (`~/.deeplake`, projection path, graph-ignore, segments) | ✅ | `src/projection/format.ts:13`, `src/registration/ignore.ts:44,47` | Verified by inspection + existing suites; no explicit AC-8-named assertion (Suggestion) | +| a-AC-1 | Default root `/.apiary`, state dir `/.apiary/nectar`, never cwd | ✅ | `src/apiary-root.ts:44-69` | `test/apiary-root.test.ts` a-AC-1 + the absolute-only security test | +| a-AC-2 | `APIARY_HOME` flows to all four adoption sites; blank treated as unset | ✅ | `src/config.ts:112-115`, `src/config-file.ts:76-79`, `src/telemetry/db.ts:40-43`, `src/telemetry-usage/emit.ts:169-172` | a-AC-2 tests in apiary-root + config suites | +| a-AC-3 | Linux XDG only when explicitly set; skipped on darwin/win32 | ✅ | `src/apiary-root.ts:53-57` | `test/apiary-root.test.ts` a-AC-3 | +| a-AC-4 | `NECTAR_RUNTIME_DIR` precedence preserved for runtimeDir/pid/lock | ✅ | `src/config.ts:112-115,124-128` | `test/apiary-root.test.ts` a-AC-4 | +| a-AC-5 | Defaults under `~/.apiary/nectar/`; no stray legacy derivation outside the migration/fallback modules | ✅ | `src/apiary-root.ts:72-74` (single source), grep-verified | fixes.test.ts + telemetry-usage.test.ts a-AC-5 tests; `doctor-registry.ts` uses `legacyRuntimeDir` per the c-AC-3 window contract (deliberate) | +| a-AC-6 | Existing test seams behave identically | ✅ | seams unchanged (`options.dir`, `deps.dir`, `overrides.runtimeDir`, `telemetryDbPathForRuntimeDir`) | Proven by the full pre-existing suite passing against the new defaults | +| b-AC-1 | All four files migrate with identical contents; legacy gone; marker written | ✅ | `src/state-migration.ts:118-183` | named test | +| b-AC-2 | Re-run performs no writes / no overwrites | ✅ | `src/state-migration.ts:127-129,136` | named test | +| b-AC-3 | Failed copy leaves legacy intact, no temp as final name, boot survives, retry works | ✅ | `src/state-migration.ts:85-96,138-162` | named test | +| b-AC-4 | Live legacy pid refuses boot; stale legacy pid proceeds | ✅ | `src/state-migration.ts:48-58`, wired at `src/daemon.ts:839-844` | named test | +| b-AC-5 | CLI/config readers fall back transparently, never migrate | ✅ | `src/state-migration.ts:28-36`, `src/cli.ts:664-670` | named test | +| b-AC-6 | Allow-list only; non-nectar legacy files byte-identical | ✅ | `src/state-migration.ts:11-16,133-136` | named test | +| b-AC-7 | Partial migration completes; readers resolve each file from wherever it is | ✅ | `src/state-migration.ts:133-141` | named test | +| c-AC-1 | Plist log paths under `~/.apiary/nectar/logs`; dir created; no `.honeycomb` literal | ✅ | `src/service/templates.ts:31-34,86-90`, `src/service/index.ts:228-233` | named tests | +| c-AC-2 | `APIARY_HOME` at install: log dir under the pinned root; unit carries the env pin | ✅ | `src/service/platform.ts:71-107,230-234`, `src/service/templates.ts:88-116,141-165` | named tests across templates/platform/index suites | +| c-AC-3 | Fresh install entry: new paths; file per window contract; other entries preserved | ✅ | `src/doctor-registry.ts:111-116,261-275` | named tests | +| c-AC-4 | Upgrade refresh idempotent; malformed registry fails loudly, not clobbered | ✅ | `src/state-migration.ts:165-178` | named tests; see Warning on the boot-abort consequence | +| c-AC-5 | Windows LocalSystem opt-in pins the installing user's root | ✅ | `src/service/argv.ts:79-91`, `src/service/templates.ts:177-199` | named tests (argv + templates + index) | +| c-AC-6 | Doctor probe succeeds end-to-end during the window | 🟦 | n/a (cross-repo) | Deferred to manual/integration verification per the AC's own text; not exercised in this repo (Warning). 2026-07-04: the orchestrator accepted this deferral for the migration window (remediation pass); no code change required. | +| NG (index) | `~/.deeplake/`, projection path, graph-ignore, segments untouched; fleet-shared files not moved by nectar | ✅ | `src/state-migration.ts:11-16` allow-list | b-AC-6 test + inspection | + +## Files Changed + +Implementation files relevant to PRD-020 (the same working tree also carries PRD-019; see that report): + +- `library/knowledge/private/architecture/ADR-0005-fleet-directory-ownership-and-neutral-state-root.md` (M), mirror re-synced with the resolved decisions + absolute-only amendment +- `src/apiary-root.ts` (A), the `resolveApiaryRoot` / `nectarStateDir` / `legacyRuntimeDir` helper implementing the ADR chain (absolute-only env roots) +- `src/cli.ts` (M), `review-matches` pending-reviews fallback read; daemon wiring (see PRD-019 report) +- `src/config-file.ts` (M), default dir via `nectarStateDir`; legacy fallback read via `resolveStateReadPath` +- `src/config.ts` (M), `resolveConfig` default runtimeDir becomes `nectarStateDir()`; `RUNTIME_DIR_NAME` re-documented as the legacy name +- `src/daemon.ts` (M), boot ordering: legacy-pid guard, migration, telemetry path re-resolution, pending-reviews fallback (plus PRD-019 wiring) +- `src/doctor-registry.ts` (M), `defaultDoctorRegistryPath` implements the ADR registry window contract +- `src/index.ts` (M), exports the apiary-root helper surface +- `src/service/argv.ts` (M), `sc` binPath carries `APIARY_HOME` via cmd wrapper when pinned +- `src/service/index.ts` (M), log/staging dirs derive from the resolved state dir +- `src/service/platform.ts` (M), `ServiceEnvironment`/`ServicePlan` carry the optional installer-pinned `apiaryHome` +- `src/service/templates.ts` (M), `launchdLogDir` under `/logs`; env pin rendered into launchd/systemd/schtasks units +- `src/state-migration.ts` (A), the one-time additive migration, legacy-pid guard, `resolveStateReadPath`, and the c-AC-4 registry refresh +- `src/telemetry/db.ts` (M), `defaultTelemetryDbPath` via `nectarStateDir` +- `src/telemetry-usage/emit.ts` (M), ledger fallback read + new-root-first `install-id` read +- `test/apiary-root.test.ts` (A), a-AC-1..4 + the absolute-only security regression +- `test/config.test.ts` (M), a-AC-2 resolveConfig adoption +- `test/doctor-registry.test.ts` (M), c-AC-3 write-target pair +- `test/fixes.test.ts` (M), a-AC-5 config-file fallback pair +- `test/service-argv.test.ts` (M), c-AC-5 sc binPath pin +- `test/service-index.test.ts` (M), c-AC-2 install render + updated log-dir assertions +- `test/service-platform.test.ts` (M), c-AC-2 plan carry +- `test/service-templates.test.ts` (M), c-AC-1 no-literal scan, c-AC-2 env pins, c-AC-5 task XML +- `test/state-migration.test.ts` (A), b-AC-1..7 + c-AC-4 pair +- `test/telemetry-usage.test.ts` (M), a-AC-5 install-id ordering pair +- `test/telemetry/db.test.ts` (M), default path under `~/.apiary/nectar` diff --git a/src/active-projects-runtime.ts b/src/active-projects-runtime.ts new file mode 100644 index 0000000..370364a --- /dev/null +++ b/src/active-projects-runtime.ts @@ -0,0 +1,82 @@ +/** + * The active-projects reconcile driver (PRD-019a). + * + * Ties the pure resolution ({@link resolveActiveProjects}) to the stateful + * {@link ProjectSupervisor} and the `/health` slice, and drives a reconcile on + * the daemon's existing poll cadence plus on demand (the 019b toggle API calls + * {@link ActiveProjectsController.reconcileNow}). No new watch dependency: it + * reuses the injected timer seam the rest of the daemon uses, so it is + * deterministic under a manual clock in tests. + */ +import { PollLoop, realTimer, type Timer } from "./poll-loop.js"; +import { ProjectSupervisor, type ProjectContextFactory, type ReconcileOutcome } from "./project-supervisor.js"; +import { activeProjectsHealth, type ActiveProjectResolution } from "./hive-graph/active-projects.js"; +import type { HealthBody } from "./health.js"; + +export interface ActiveProjectsControllerDeps { + /** Resolve the current active set (reads the `~/.deeplake` bindings + the brooding-state file). */ + resolve(): ActiveProjectResolution; + /** Build a running brood + watch context for a project. */ + readonly factory: ProjectContextFactory; + /** Apply the computed `/health` active-projects slice. */ + setHealth(slice: HealthBody["activeProjects"]): void; + /** Reconcile cadence in ms (default: the daemon's poll interval). */ + readonly intervalMs: number; + /** Injected timer (tests). Default: the real timer. */ + readonly timer?: Timer; + /** Observe a context start/stop failure. Default: no-op. */ + readonly onError?: (scope: "start" | "stop", projectId: string, err: unknown) => void; +} + +export class ActiveProjectsController { + readonly supervisor: ProjectSupervisor; + private readonly deps: ActiveProjectsControllerDeps; + private readonly loop: PollLoop; + private started = false; + + constructor(deps: ActiveProjectsControllerDeps) { + this.deps = deps; + this.supervisor = new ProjectSupervisor({ + factory: deps.factory, + ...(deps.onError !== undefined ? { onError: deps.onError } : {}), + }); + this.loop = new PollLoop({ + tick: async () => { + const outcome = await this.reconcileNow(); + return outcome.started.length > 0 || outcome.stopped.length > 0; + }, + floorMs: Math.max(1, deps.intervalMs), + timer: deps.timer ?? realTimer, + }); + } + + /** Publish the current resolution to `/health` without touching the supervisor (dormant snapshot). */ + publishHealth(): void { + const resolution = this.deps.resolve(); + this.deps.setHealth(activeProjectsHealth(resolution, (id) => this.supervisor.watcherStateFor(id))); + } + + /** Resolve the active set, reconcile the supervisor to it, and refresh the `/health` slice. */ + async reconcileNow(): Promise { + const resolution = this.deps.resolve(); + const outcome = await this.supervisor.reconcile(resolution.active); + this.deps.setHealth(activeProjectsHealth(resolution, (id) => this.supervisor.watcherStateFor(id))); + return outcome; + } + + /** Arm the reconcile poll loop (the loop fires an immediate first tick at delay 0). Idempotent. */ + start(): void { + if (this.started) return; + this.started = true; + this.loop.start(); + } + + /** Disarm the loop, drain any in-flight reconcile, then stop every running context (a-AC-7). */ + async stop(): Promise { + this.loop.stop(); + await this.loop.whenIdle(); + await this.supervisor.stopAll(); + this.started = false; + this.deps.setHealth(activeProjectsHealth(this.deps.resolve(), () => "stopped")); + } +} diff --git a/src/api/loopback-client.ts b/src/api/loopback-client.ts index cd3ad04..5a18b76 100644 --- a/src/api/loopback-client.ts +++ b/src/api/loopback-client.ts @@ -15,6 +15,7 @@ */ import { request } from "node:http"; import type { HiveGraphSearchResult } from "../hive-graph/search-types.js"; +import type { ProjectsView } from "../projects-control.js"; /** Raised when the loopback request cannot reach the daemon (not running / refused / timed out). */ export class DaemonUnreachableError extends Error { @@ -103,3 +104,79 @@ export function searchViaDaemon(options: LoopbackSearchOptions): Promise( + host: string, + port: number, + method: "GET" | "POST", + path: string, + payload: string | undefined, + timeoutMs: number, +): Promise { + return new Promise((resolve, reject) => { + const headers: Record = {}; + if (payload !== undefined) { + headers["content-type"] = "application/json; charset=utf-8"; + headers["content-length"] = Buffer.byteLength(payload); + } + const req = request({ host, port, method, path, headers, timeout: timeoutMs }, (res) => { + const chunks: Buffer[] = []; + res.on("data", (c: Buffer) => chunks.push(c)); + res.on("end", () => { + const text = Buffer.concat(chunks).toString("utf8"); + const status = res.statusCode ?? 0; + if (status < 200 || status >= 300) { + reject(new DaemonSearchError(`daemon returned HTTP ${status}: ${text.slice(0, 200)}`, status)); + return; + } + try { + resolve(JSON.parse(text) as T); + } catch { + reject(new DaemonSearchError("daemon returned a non-JSON response", status)); + } + }); + }); + req.on("error", (err: NodeJS.ErrnoException) => reject(new DaemonUnreachableError(err.message))); + req.on("timeout", () => { + req.destroy(new DaemonUnreachableError(`request to ${host}:${port} timed out after ${timeoutMs}ms`)); + }); + if (payload !== undefined) req.end(payload); + else req.end(); + }); +} + +export interface LoopbackDaemonTarget { + readonly host: string; + readonly port: number; + readonly timeoutMs?: number; +} + +/** GET the active-project + brooding view from the daemon (`nectar projects`, PRD-019b). */ +export function projectsViaDaemon(target: LoopbackDaemonTarget): Promise { + return daemonJsonRequest( + target.host, + target.port, + "GET", + "/api/hive-graph/projects", + undefined, + target.timeoutMs ?? DEFAULT_TIMEOUT_MS, + ); +} + +/** The body accepted by `POST /api/hive-graph/projects/brooding` (PRD-019b). */ +export type BroodingToggleBody = + | { readonly projectId: string; readonly brooding: "on" | "off" } + | { readonly global: "on" | "paused" }; + +/** POST a brooding toggle to the daemon and return the new view (`nectar brooding`, PRD-019b). */ +export function setBroodingViaDaemon(target: LoopbackDaemonTarget, body: BroodingToggleBody): Promise { + return daemonJsonRequest( + target.host, + target.port, + "POST", + "/api/hive-graph/projects/brooding", + JSON.stringify(body), + target.timeoutMs ?? DEFAULT_TIMEOUT_MS, + ); +} diff --git a/src/api/projects-api.ts b/src/api/projects-api.ts new file mode 100644 index 0000000..ed518e2 --- /dev/null +++ b/src/api/projects-api.ts @@ -0,0 +1,149 @@ +/** + * `mountProjectsApi` - the projects + brooding-control endpoints on the + * `/api/hive-graph` group (PRD-019b), consumed by Hive's dashboard (019c). + * + * - `GET /api/hive-graph/projects` - the active-project set joined with + * the brooding state and each project's watcher slice (read-only). + * - `POST /api/hive-graph/projects/brooding` - set a single project's brooding + * (`{ projectId, brooding }`) or the global switch (`{ global }`). On success it + * persists, triggers an immediate reconcile, then returns the new view. + * + * Both routes sit under the already-mounted `/api/hive-graph` permission gate + * (PRD-008a / PRD-018j) and are loopback-only. The body is hand-validated + * (nectar is zero-runtime-dependency, so no zod at runtime); an invalid body is a + * 400, a persist (disk) failure is a redacted 500 that leaves the prior state + * intact and does NOT reconcile (b-AC-6). + */ +import type { RouteContext, RouteGroup, RouteResponse } from "./router.js"; +import { HIVE_GRAPH_GROUP, MalformedJsonError } from "./router.js"; +import type { ProjectsView } from "../projects-control.js"; +import type { ProjectBrooding, GlobalBrooding } from "../registration/brooding-state.js"; + +/** The `group()` accessor surface `mountProjectsApi` needs. */ +export interface RouteGroupProvider { + group(path: string): RouteGroup | undefined; +} + +/** The injected mechanics for the projects endpoints. */ +export interface MountProjectsOptions { + /** Build the current read view (resolve bindings + brooding state + watcher liveness). */ + view(): ProjectsView; + /** Persist a per-project brooding change. Throws on a write failure (mapped to 500, prior state intact). */ + setProject(projectId: string, brooding: ProjectBrooding): void; + /** Persist the global switch. Throws on a write failure. */ + setGlobal(global: GlobalBrooding): void; + /** Trigger an immediate active-set reconcile after a successful persist. */ + reconcile(): Promise; + /** + * The brooding-state file path included in the REDACTED persist-failure body + * (b-AC-6): the caller learns WHICH file could not be written, never the raw + * OS error internals. Absent -> the body carries the stable reason only. + */ + readonly stateFilePath?: string; + /** + * Observe the RAW persist failure server-side (for the daemon log). The HTTP + * body never carries the raw error (b-AC-6 redaction); this seam keeps the + * diagnostics from being lost. Default: no-op. + */ + readonly onPersistError?: (err: unknown) => void; +} + +function errorReason(err: unknown): string { + return err instanceof Error ? err.message : String(err); +} + +/** A validated brooding toggle request: one of a per-project change or a global change. */ +export type BroodingToggleRequest = + | { readonly kind: "project"; readonly projectId: string; readonly brooding: ProjectBrooding } + | { readonly kind: "global"; readonly global: GlobalBrooding }; + +/** Thrown by {@link parseBroodingToggle} for a body that is neither a valid project nor global change. */ +export class InvalidBroodingToggleError extends Error { + constructor(message: string) { + super(message); + this.name = "InvalidBroodingToggleError"; + } +} + +/** + * Hand-validate the toggle body. Accepts exactly one of + * `{ projectId: string, brooding: "on"|"off" }` or `{ global: "on"|"paused" }`. + * Anything else throws {@link InvalidBroodingToggleError} (mapped to 400). + */ +export function parseBroodingToggle(body: unknown): BroodingToggleRequest { + if (typeof body !== "object" || body === null || Array.isArray(body)) { + throw new InvalidBroodingToggleError("body must be a JSON object"); + } + const rec = body as Record; + const hasGlobal = rec["global"] !== undefined; + const hasProject = rec["projectId"] !== undefined || rec["brooding"] !== undefined; + if (hasGlobal && hasProject) { + throw new InvalidBroodingToggleError("provide either { global } or { projectId, brooding }, not both"); + } + if (hasGlobal) { + const global = rec["global"]; + if (global !== "on" && global !== "paused") { + throw new InvalidBroodingToggleError('global must be "on" or "paused"'); + } + return { kind: "global", global }; + } + const projectId = rec["projectId"]; + const brooding = rec["brooding"]; + if (typeof projectId !== "string" || projectId.trim() === "") { + throw new InvalidBroodingToggleError("projectId must be a non-empty string"); + } + if (brooding !== "on" && brooding !== "off") { + throw new InvalidBroodingToggleError('brooding must be "on" or "off"'); + } + return { kind: "project", projectId: projectId.trim(), brooding }; +} + +/** + * Attach the projects + brooding-control handlers to the `/api/hive-graph` group. + * Safe to call against a daemon whose group is unknown (no-op attach). Called + * once after `assembleDaemon(...)`, alongside `mountHiveGraphApi`. + */ +export function mountProjectsApi(daemon: RouteGroupProvider, options: MountProjectsOptions): void { + const group = daemon.group(HIVE_GRAPH_GROUP); + if (group === undefined) return; + + group.get("/projects", (ctx): RouteResponse => { + try { + return ctx.json(options.view()); + } catch (err: unknown) { + return ctx.json({ error: "projects_read_failed", reason: errorReason(err) }, 500); + } + }); + + group.post("/projects/brooding", async (ctx): Promise => { + let request: BroodingToggleRequest; + try { + request = parseBroodingToggle(ctx.body()); + } catch (err: unknown) { + if (err instanceof MalformedJsonError) return ctx.json({ error: "invalid_json", reason: errorReason(err) }, 400); + if (err instanceof InvalidBroodingToggleError) return ctx.json({ error: "invalid_request", reason: errorReason(err) }, 400); + return ctx.json({ error: "invalid_request", reason: errorReason(err) }, 400); + } + // Persist first; on a write failure the prior state is intact and we do NOT + // reconcile against a half-written file (b-AC-6). + try { + if (request.kind === "project") options.setProject(request.projectId, request.brooding); + else options.setGlobal(request.global); + } catch (err: unknown) { + // b-AC-6: the error is REDACTED. The body carries a stable reason plus the + // state-file path only, never the raw OS error text (errno strings embed + // local paths and OS internals). The raw failure routes to the server-side + // observer so diagnostics land in the daemon log, not the HTTP response. + options.onPersistError?.(err); + const where = options.stateFilePath !== undefined ? ` (${options.stateFilePath})` : ""; + return ctx.json({ error: "persist_failed", reason: `could not persist the brooding state${where}` }, 500); + } + // Reconcile so the change takes effect within this call (b-AC-3/4/5). + try { + await options.reconcile(); + } catch (err: unknown) { + return ctx.json({ error: "reconcile_failed", reason: errorReason(err) }, 500); + } + return ctx.json(options.view()); + }); +} diff --git a/src/apiary-root.ts b/src/apiary-root.ts new file mode 100644 index 0000000..fab94b5 --- /dev/null +++ b/src/apiary-root.ts @@ -0,0 +1,74 @@ +import { homedir } from "node:os"; +import { join, win32 } from "node:path"; + +/** Legacy runtime directory basename retained for migration/fallback reads. */ +export const LEGACY_RUNTIME_DIR_NAME = ".honeycomb"; + +/** Fleet root basename when neither APIARY_HOME nor Linux XDG are set. */ +export const APIARY_ROOT_DIR_NAME = ".apiary"; + +/** Nectar's per-product state directory basename under the fleet root. */ +export const NECTAR_STATE_DIR_NAME = "nectar"; + +export interface ResolveApiaryRootOptions { + readonly platform?: NodeJS.Platform; + readonly home?: string; +} + +function nonBlank(value: string | undefined): string | undefined { + if (value === undefined) return undefined; + const trimmed = value.trim(); + if (trimmed === "") return undefined; + return trimmed; +} + +/** + * Env roots are honored only when ABSOLUTE (fleet security rule, 2026-07-04; the + * XDG Base Directory spec also requires ignoring relative values). Honoring a + * relative value would anchor the fleet root, and everything derived from it, + * on process.cwd(), the exact footgun ADR-0005 exists to prevent. + * `win32.isAbsolute` accepts `/x`, `\x`, and `C:\x`, a strict superset of the + * posix check, so a relative value is never mistaken for absolute on any host. + */ +function isAbsoluteRoot(value: string): boolean { + return win32.isAbsolute(value); +} + +/** + * Resolve the fleet root per ADR-0005's canonical chain. + * + * 1) APIARY_HOME when set, non-blank, and absolute. + * 2) Linux-only XDG_STATE_HOME/apiary when explicitly set, non-blank, and absolute. + * 3) /.apiary on every platform otherwise. + */ +export function resolveApiaryRoot( + env: NodeJS.ProcessEnv = process.env, + options: ResolveApiaryRootOptions = {}, +): string { + const configuredApiaryHome = nonBlank(env.APIARY_HOME); + if (configuredApiaryHome !== undefined && isAbsoluteRoot(configuredApiaryHome)) { + return configuredApiaryHome; + } + + const platform = options.platform ?? process.platform; + const configuredXdgStateHome = nonBlank(env.XDG_STATE_HOME); + if (platform === "linux" && configuredXdgStateHome !== undefined && isAbsoluteRoot(configuredXdgStateHome)) { + return join(configuredXdgStateHome, "apiary"); + } + + const home = options.home ?? homedir(); + return join(home, APIARY_ROOT_DIR_NAME); +} + +/** Resolve nectar's per-product runtime state directory under the fleet root. */ +export function nectarStateDir( + env: NodeJS.ProcessEnv = process.env, + options: ResolveApiaryRootOptions = {}, +): string { + return join(resolveApiaryRoot(env, options), NECTAR_STATE_DIR_NAME); +} + +/** Resolve the legacy ~/.honeycomb runtime directory used during migration. */ +export function legacyRuntimeDir(home: string = homedir()): string { + return join(home, LEGACY_RUNTIME_DIR_NAME); +} diff --git a/src/cli.ts b/src/cli.ts index e7abdfe..76c8965 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -29,11 +29,18 @@ import { realpathSync } from "node:fs"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { createInterface } from "node:readline/promises"; -import { assembleDaemon, type AssembledDaemon, type AssembleOptions, type BootProjectionLoad } from "./daemon.js"; +import { assembleDaemon, type AssembledDaemon, type AssembleOptions } from "./daemon.js"; import { resolveConfig } from "./config.js"; import { mountHiveGraphApi } from "./api/hive-graph-api.js"; import { buildHiveGraphApiOptions } from "./api/daemon-api-wiring.js"; -import { searchViaDaemon, DaemonUnreachableError, DaemonSearchError } from "./api/loopback-client.js"; +import { + searchViaDaemon, + projectsViaDaemon, + setBroodingViaDaemon, + DaemonUnreachableError, + DaemonSearchError, +} from "./api/loopback-client.js"; +import type { ProjectsView } from "./projects-control.js"; import type { HiveGraphSearchResult, HiveGraphHit } from "./hive-graph/search-types.js"; import { createServiceModule, serviceStatus } from "./service/index.js"; import { registerWithDoctor } from "./doctor-registry.js"; @@ -47,6 +54,20 @@ import { broodPrereqsFromEnv, formatFirstRunGuidance } from "./brood-prereqs.js" import { resolveNectarTunables } from "./config-file.js"; import { resolveProjectScope, type ProjectScopeSource } from "./hive-graph/project-scope.js"; import { createDiskRegistrationFs } from "./registration/disk-fs.js"; +import { createSharedIgnore, type IgnorePredicate } from "./registration/ignore.js"; +import { createProjectContext } from "./registration/project-context.js"; +import type { WatcherState } from "./registration/fs-watch.js"; +import type { RunningContext } from "./project-supervisor.js"; +import type { ResolvedProject } from "./hive-graph/active-projects.js"; +import { + readActiveProjects, + buildProjectsView, + persistProjectBrooding, + persistGlobalBrooding, + type ProjectsControlOptions, +} from "./projects-control.js"; +import { broodingStatePath } from "./registration/brooding-state.js"; +import { mountProjectsApi } from "./api/projects-api.js"; import { StoreBridge } from "./registration/store-bridge.js"; import { runPrune } from "./registration/prune-cli.js"; import { runReviewMatches, type ReviewDecision } from "./registration/review-cli.js"; @@ -55,9 +76,8 @@ import { type PendingReviewStore, type PendingReviewCandidate, } from "./registration/review-store.js"; -import { rebuildProjectionAsync, projectionFinalPath, ProjectionWriter } from "./projection/write.js"; -import { DEFAULT_PROJECTION_REL_PATH } from "./projection/format.js"; -import type { InheritRow } from "./projection/inherit.js"; +import { resolveStateReadPath } from "./state-migration.js"; +import { rebuildProjectionAsync, ProjectionWriter } from "./projection/write.js"; import { resolvePortkeyConfig, type PortkeyEnabled } from "./portkey/config.js"; import { activeEmbedModelId, resolveEmbeddingsConfig, validateEmbedDimension } from "./embeddings/config.js"; import { resolveEmbedProvider, type EmbedProvider } from "./embeddings/provider.js"; @@ -70,8 +90,6 @@ import { parseBroodArgs, planBrood, formatDryRunReport, - discoverFiles, - prepareFiles, runBroodAsync, type BroodConfig, type BroodRunOptions, @@ -92,6 +110,11 @@ Usage: nectar search [flags] Manual hive-graph search (PRD-012). Thin loopback client of the daemon search endpoint (POST /api/hive-graph/search). Requires a running 'nectar daemon'. Flags: --limit N, --json + nectar projects [--json] List the active projects + brooding state (PRD-019). Thin loopback + client of GET /api/hive-graph/projects. Requires a running daemon. + nectar brooding [--project |--all] [--global-pause|--global-resume] + Turn brooding on/off per project or globally (PRD-019). Thin loopback + client of POST /api/hive-graph/projects/brooding. Requires a running daemon. nectar prune [--confirm] Prune long-missing nectars from the durable store (PRD-006d) nectar review-matches Review low-confidence step-4 matches against the durable store (PRD-006d) nectar rebuild-projection Regenerate .honeycomb/nectars.json from Deep Lake (PRD-011) @@ -182,6 +205,22 @@ async function runServiceStatus(): Promise { return status === "unknown" ? 1 : 0; } +/** + * PRD-019d: build the ONE shared ignore predicate (segments UNION graph-ignore + * UNION gitignore semantics, with the git-absent `.gitignore` parser fallback) + * for a CLI command's resolved root, so every CLI discovery path excludes + * exactly the set the daemon watch path does - including on a non-git root. + */ +function sharedIgnoreFor(root: string): IgnorePredicate { + return createSharedIgnore(root).isIgnored; +} + +/** Read a running project's watcher liveness off the daemon's `/health` active-projects slice (PRD-019b view). */ +function daemonWatcherState(daemon: AssembledDaemon, projectId: string): WatcherState { + const entry = daemon.health.snapshot().activeProjects.projects.find((p) => p.projectId === projectId); + return (entry?.watcher ?? "stopped") as WatcherState; +} + /** Read an env var, treating unset OR blank/whitespace-only as absent (mirrors config.ts). */ function cliEnvStr(name: string): string | undefined { const raw = process.env[name]; @@ -299,11 +338,13 @@ function runBroodDryRun(): number { ); return 1; } + const isIgnored = sharedIgnoreFor(ctx.projectRoot); const config: BroodConfig = { store: new InMemoryHiveGraphStore(), tenancy: ctx.tenancy, root: ctx.projectRoot, - fs: createDiskRegistrationFs(ctx.projectRoot), + isIgnored, + fs: createDiskRegistrationFs(ctx.projectRoot, isIgnored), }; const plan = planBrood(config); process.stdout.write( @@ -539,12 +580,14 @@ async function runBroodMutating(options: BroodRunOptions): Promise { return 1; } const { embedProvider, embedModelId } = resolveCliBroodEmbedDeps(); + const isIgnored = sharedIgnoreFor(ctx.projectRoot); return runBroodMutatingVerb({ config: { store: ctx.store, tenancy: ctx.tenancy, root: ctx.projectRoot, - fs: createDiskRegistrationFs(ctx.projectRoot), + isIgnored, + fs: createDiskRegistrationFs(ctx.projectRoot, isIgnored), }, deps: { portkey, embedProvider, embedModelId }, options, @@ -598,7 +641,7 @@ async function withDurableBridge( async function runPruneCommand(args: readonly string[]): Promise { const confirm = args.includes("--confirm"); return withDurableBridge("prune", async (bridge, ctx) => { - const fs = createDiskRegistrationFs(ctx.projectRoot); + const fs = createDiskRegistrationFs(ctx.projectRoot, sharedIgnoreFor(ctx.projectRoot)); return runPruneVerb({ store: bridge, tenancy: ctx.tenancy, @@ -616,7 +659,11 @@ async function runPruneCommand(args: readonly string[]): Promise { */ async function runReviewMatchesCommand(): Promise { const config = resolveConfig(); - const reviews = new FilePendingReviewStore(join(config.runtimeDir, "pending-reviews.json")); + const hasRuntimeDirOverride = (process.env["NECTAR_RUNTIME_DIR"] ?? "").trim() !== ""; + const reviewsPath = hasRuntimeDirOverride + ? join(config.runtimeDir, "pending-reviews.json") + : resolveStateReadPath("pending-reviews.json", { runtimeDir: config.runtimeDir }); + const reviews = new FilePendingReviewStore(reviewsPath); const decider = interactiveReviewDecider(); try { return await withDurableBridge("review-matches", (bridge, ctx) => @@ -760,55 +807,163 @@ async function runSearchCommand(args: readonly string[]): Promise { } } +// ── PRD-019b: `nectar projects` + `nectar brooding` ───────────────────────── + +/** Render the projects view as a human-readable table (default, non-`--json`). */ +export function renderProjectsTable(view: ProjectsView): string { + const lines: string[] = []; + lines.push(`global brooding: ${view.globalBrooding}`); + if (view.projects.length === 0) { + lines.push("No active projects. Add one by selecting a directory in the Hive dashboard."); + } else { + lines.push(`${view.projects.length} project${view.projects.length === 1 ? "" : "s"}:`); + for (const p of view.projects) { + const label = p.name.trim() !== "" ? `${p.name} (${p.projectId})` : p.projectId; + lines.push(` - ${label}`); + lines.push(` path: ${p.path}`); + lines.push(` brooding: ${p.brooding} watcher: ${p.watcher}`); + } + } + return `${lines.join("\n")}\n`; +} + +/** `nectar projects [--json]`: the read side of `GET /api/hive-graph/projects`. */ +async function runProjectsCommand(args: readonly string[]): Promise { + const json = args.includes("--json"); + const config = resolveConfig(); + try { + const view = await projectsViaDaemon({ host: config.host, port: config.port }); + if (json) process.stdout.write(`${JSON.stringify(view)}\n`); + else process.stdout.write(renderProjectsTable(view)); + return 0; + } catch (err: unknown) { + if (err instanceof DaemonUnreachableError) { + process.stderr.write( + `nectar projects: the nectar daemon is not reachable on ${config.host}:${config.port} (${err.message}).\n` + + "Start it with 'nectar daemon'.\n", + ); + return 2; + } + process.stderr.write(`nectar projects: ${err instanceof Error ? err.message : String(err)}\n`); + return 1; + } +} + +/** The parsed `nectar brooding` invocation (PRD-019b). */ +export type BroodingInvocation = + | { readonly kind: "errors"; readonly errors: readonly string[] } + | { readonly kind: "global"; readonly global: "on" | "paused" } + | { readonly kind: "project"; readonly projectId: string; readonly brooding: "on" | "off" } + | { readonly kind: "all"; readonly brooding: "on" | "off" }; + /** - * Build the boot projection load seam for the live daemon (PRD-011b AC-6): if a - * project context resolves, the daemon validates `.honeycomb/nectars.json` on - * boot and inherits hash-matched files into the durable Deep Lake store. All - * work is deferred to lazy providers so nothing scans disk or hits the network - * until AFTER the daemon is accepting requests; fail-soft throughout (a missing - * credentials file just skips the pre-warm). Returns undefined when no context - * resolves, so `nectar daemon` still starts on a bare machine. + * Parse `nectar brooding [--project |--all]` and the global flags + * `--global-pause` / `--global-resume`. The global flags are standalone (no + * on|off positional); the on|off form requires exactly one of `--project ` + * or `--all`. */ -function resolveBootProjection(): BootProjectionLoad | undefined { - const ctx = resolveProjectionContext(); - if (!ctx.ok) return undefined; - const { store, tenancy, projectRoot } = ctx; - return { - tenancy, - filePath: projectionFinalPath(projectRoot, DEFAULT_PROJECTION_REL_PATH), - diskHashes: () => { - const discovery = discoverFiles({ root: projectRoot, fs: createDiskRegistrationFs(projectRoot) }); - const prepared = prepareFiles(createDiskRegistrationFs(projectRoot), discovery.files); - return new Map(prepared.map((p) => [p.file.relPath, p.contentHash] as const)); - }, - existingNectars: async () => { - const latest = await store.listLatestVersions(tenancy); - return new Set(latest.map((lv) => lv.identity.nectar)); - }, - write: async (rows: readonly InheritRow[]) => { - for (const row of rows) { - if ((await store.getIdentity(row.identity.nectar)) === undefined) { - await store.insertIdentity(row.identity); - } - await store.appendVersion(row.version); +export function parseBroodingArgs(args: readonly string[]): BroodingInvocation { + const errors: string[] = []; + let positional: string | undefined; + let projectId: string | undefined; + let all = false; + let globalPause = false; + let globalResume = false; + + for (let i = 0; i < args.length; i++) { + const arg = args[i] ?? ""; + if (arg === "--all") { + all = true; + } else if (arg === "--global-pause") { + globalPause = true; + } else if (arg === "--global-resume") { + globalResume = true; + } else if (arg === "--project") { + const raw = args[i + 1]; + if (raw === undefined || raw.startsWith("--")) errors.push("--project requires a value"); + else { + projectId = raw; + i++; } - }, - }; -} + } else if (arg.startsWith("--project=")) { + projectId = arg.slice("--project=".length); + } else if (arg.startsWith("--")) { + errors.push(`unknown flag '${arg}'`); + } else if (positional === undefined) { + positional = arg; + } else { + errors.push(`unexpected argument '${arg}'`); + } + } -/** The resolved-and-ok shape of {@link resolveProjectionContext}. */ -type ResolvedContext = Extract, { readonly ok: true }>; + if (globalPause && globalResume) errors.push("use only one of --global-pause / --global-resume"); + const isGlobal = globalPause || globalResume; + + if (isGlobal) { + if (positional !== undefined) errors.push("the global flags do not take an on|off argument"); + if (projectId !== undefined || all) errors.push("the global flags cannot combine with --project / --all"); + if (errors.length > 0) return { kind: "errors", errors }; + return { kind: "global", global: globalPause ? "paused" : "on" }; + } -/** Resolve the Deep Lake project context, swallowing a missing-creds error into `undefined`. */ -function safeResolveContext(): ResolvedContext | undefined { + if (positional !== "on" && positional !== "off") { + errors.push("expected 'on' or 'off': nectar brooding [--project |--all]"); + } + if (projectId !== undefined && all) errors.push("use only one of --project / --all"); + if (projectId === undefined && !all) errors.push("specify --project or --all"); + if (errors.length > 0) return { kind: "errors", errors }; + + const brooding = positional as "on" | "off"; + if (all) return { kind: "all", brooding }; + return { kind: "project", projectId: projectId as string, brooding }; +} + +/** `nectar brooding ...`: the write side of `POST /api/hive-graph/projects/brooding`. */ +async function runBroodingCommand(args: readonly string[]): Promise { + const invocation = parseBroodingArgs(args); + if (invocation.kind === "errors") { + for (const err of invocation.errors) process.stderr.write(`nectar brooding: ${err}\n`); + return 2; + } + const config = resolveConfig(); + const target = { host: config.host, port: config.port }; try { - const result = resolveProjectionContext(); - return result.ok ? result : undefined; - } catch { - return undefined; + let view: ProjectsView; + if (invocation.kind === "global") { + view = await setBroodingViaDaemon(target, { global: invocation.global }); + } else if (invocation.kind === "project") { + view = await setBroodingViaDaemon(target, { projectId: invocation.projectId, brooding: invocation.brooding }); + } else { + // --all: set every currently-bound project to the requested state. + const current = await projectsViaDaemon(target); + view = current; + for (const p of current.projects) { + view = await setBroodingViaDaemon(target, { projectId: p.projectId, brooding: invocation.brooding }); + } + } + process.stdout.write(renderProjectsTable(view)); + return 0; + } catch (err: unknown) { + if (err instanceof DaemonUnreachableError) { + process.stderr.write( + `nectar brooding: the nectar daemon is not reachable on ${config.host}:${config.port} (${err.message}).\n` + + "Start it with 'nectar daemon'.\n", + ); + return 2; + } + process.stderr.write(`nectar brooding: ${err instanceof Error ? err.message : String(err)}\n`); + return 1; } } +// The single-root `resolveBootProjection` boot pre-warm was removed with the +// PRD-019a multi-root rewire: the PRD-011b projection load + inherit now runs +// PER PROJECT inside `createProjectContext.start()` (`registration/project-context.ts`), +// so each activated root validates its own `.honeycomb/nectars.json`. + +/** The resolved-and-ok shape of {@link resolveProjectionContext}. */ +type ResolvedContext = Extract, { readonly ok: true }>; + /** A metrics sink that delegates to the daemon's CURRENT telemetry (fresh per start()), for the live build brood path. */ function daemonMetricsProxy(daemon: AssembledDaemon): PipelineMetricsSink { return { @@ -840,7 +995,7 @@ function buildLiveDurableWiring( ctx: ResolvedContext, portkey: PortkeyEnabled, ): LiveDurableWiring { - const fs = createDiskRegistrationFs(ctx.projectRoot); + const fs = createDiskRegistrationFs(ctx.projectRoot, sharedIgnoreFor(ctx.projectRoot)); const readContent: ContentReader = { read: (path: string): string | null => { const stat = fs.statPath(path); @@ -869,7 +1024,6 @@ function buildLiveDurableWiring( } async function runDaemon(): Promise { - const ctx = safeResolveContext(); const portkey = resolvePortkeyConfig(); const embeddingsConfig = resolveEmbeddingsConfig({}); // AC-018i.7: catch a wrong NECTAR_EMBEDDINGS_OUTPUT_DIMENSION at config @@ -883,67 +1037,87 @@ async function runDaemon(): Promise { }); const embedModel = activeEmbedModelId(embeddingsConfig); - let bootProjection: BootProjectionLoad | undefined; + // PRD-019a: resolve the shared Deep Lake credentials WITHOUT pinning a single + // project (the daemon is multi-root now). Creds absent -> the daemon boots, + // serves /health, and stays dormant (no durable store to brood into). + let loadedCredentials: ReturnType | undefined; try { - bootProjection = resolveBootProjection(); + loadedCredentials = loadDeepLakeCredentials(); } catch { - // fail-soft: a boot pre-warm is best-effort and never blocks the daemon start. - bootProjection = undefined; + loadedCredentials = undefined; } + // Const snapshots so the deferred factory / view closures narrow correctly. + const creds = loadedCredentials; + const durableStore = creds !== undefined ? new DeepLakeHiveGraphStore({ credentials: creds }) : undefined; - // Durable brood + enrich run live only when Deep Lake creds resolve AND a - // Portkey describe transport is configured: an LLM-less daemon genuinely - // cannot brood or enrich, so it stays dormant (honest, and no false-fail - // marking of pending rows). This is the bridge that closes the Wave D dormancy. - const live: LiveDurableWiring | undefined = - ctx !== undefined && portkey.enabled ? buildLiveDurableWiring(ctx, portkey) : undefined; - - // PRD-018k / NEC-023: resolve the brood prerequisites so a dormant daemon - // surfaces WHY on /health and in the startup log (and, on a TTY, prints the - // guided first-run steps below). PRD-018k / NEC-041: resolve the per-repo - // tunables (`~/.honeycomb/nectar.json`, env-over-file) once at boot. - const broodPrereqs = broodPrereqsFromEnv({ credentialsPresent: ctx !== undefined, credentialsPath: credentialsPath() }); + // PRD-018k / NEC-023: brood prerequisites so a dormant daemon surfaces WHY. + const broodPrereqs = broodPrereqsFromEnv({ credentialsPresent: creds !== undefined, credentialsPath: credentialsPath() }); const tunables = resolveNectarTunables(); + // PRD-019a: the active-project supervisor seam. `resolve` reads the shared + // `~/.deeplake/projects.json` bindings + the nectar-owned brooding state; the + // factory stands up one brood + watch context per active project, each rooted + // at its own bound path and scoped to its own tenancy project id. Live brood + // (describe) deps are attached only when Portkey is configured. + const controlOptions: ProjectsControlOptions = + creds !== undefined ? { expect: { org: creds.orgId, workspace: creds.workspaceId } } : {}; + const broodDeps = + creds !== undefined && portkey.enabled + ? { portkey: portkey as PortkeyEnabled, embedProvider, embedModelId: embedModel } + : undefined; + + const activeProjects = + durableStore !== undefined && creds !== undefined + ? { + resolve: () => readActiveProjects(controlOptions).resolution, + factory: (project: ResolvedProject): RunningContext => + createProjectContext({ + project, + tenancy: { orgId: creds.orgId, workspaceId: creds.workspaceId, projectId: project.projectId }, + store: durableStore, + ...(broodDeps !== undefined ? { broodDeps } : {}), + log: (line) => process.stderr.write(`${JSON.stringify({ ts: new Date().toISOString(), ...line })}\n`), + }), + } + : undefined; + + // The daemon-level enricher stays scoped to the PRIMARY active project at boot + // (the common single-project case); per-project enrichment for additional + // active projects is a documented follow-up. Absent creds/Portkey/active set + // -> the enricher stays dormant (honest). + const primary = durableStore !== undefined ? readActiveProjects(controlOptions).resolution.active[0] : undefined; + const primaryTenancy: Tenancy | undefined = + primary !== undefined && creds !== undefined + ? { orgId: creds.orgId, workspaceId: creds.workspaceId, projectId: primary.projectId } + : undefined; + const live: LiveDurableWiring | undefined = + durableStore !== undefined && creds !== undefined && primary !== undefined && primaryTenancy !== undefined && portkey.enabled + ? buildLiveDurableWiring( + { ok: true, tenancy: primaryTenancy, projectRoot: primary.path, store: durableStore, scopeSource: "binding", credentials: creds }, + portkey, + ) + : undefined; + const options: AssembleOptions = { broodPrereqs, - ...(bootProjection !== undefined ? { bootProjection } : {}), - // PRD-018b: when Deep Lake creds resolve, start the update-on-change watch - // pipeline against the durable store (the watch leg needs no LLM - it appends - // pending version rows the enricher describes later). Absent creds -> the - // watch leg stays dormant and /health says so (AC-018b.7). - ...(ctx !== undefined - ? { tenancy: ctx.tenancy, projectRoot: ctx.projectRoot, registrationStore: ctx.store } - : {}), - ...(live !== undefined && ctx !== undefined + ...(activeProjects !== undefined ? { activeProjects } : {}), + ...(live !== undefined && primaryTenancy !== undefined && primary !== undefined ? { - asyncBroodStore: ctx.store, - // AC-018i.1: stamp embed_model on auto-brood-embedded rows. - broodDepsAsync: { portkey: live.portkey, embedProvider, embedModelId: embedModel }, + tenancy: primaryTenancy, + projectRoot: primary.path, enricherStore: live.enricher, enricherCycle: { readContent: live.readContent, portkey: live.portkey, embedProvider, - // PRD-018k / NEC-041 AC-018k.6: source the redescribe threshold from - // ~/.honeycomb/nectar.json (env-over-file), falling back to the code - // default when neither the env var nor the file supplies one. ...(tunables.redescribeThreshold !== undefined ? { config: { redescribeThreshold: tunables.redescribeThreshold } } : {}), - // AC-018i.1: stamp embed_model on enricher-embedded rows. embedModel, - // AC-018g.6: re-seed the working set each cycle from the durable store. - refreshWorkingSet: () => live.enricher.refresh(ctx.tenancy), - // AC-018g.9: prior-content cache backs the cosmetic-change gate. + refreshWorkingSet: () => live.enricher.refresh(primaryTenancy), priorContentCache: createPriorContentCache(), - // AC-018g.11/.12: trigger #2 - a debounced projection write after a - // cycle that wrote descriptions, sourced from the enricher mirror. - projectionWriter: new ProjectionWriter({ projectRoot: ctx.projectRoot }), - projectionDoc: () => live.enricher.buildProjectionDoc(ctx.tenancy), - // Surface the underlying describe-batch error; without this the - // persistent-failure alert can trip with nothing in the logs to - // diagnose (2026-07-03 production soak stall). + projectionWriter: new ProjectionWriter({ projectRoot: primary.path }), + projectionDoc: () => live.enricher.buildProjectionDoc(primaryTenancy), onDescribeError: (err: unknown, paths: readonly string[]) => { process.stderr.write( `nectar enricher: describe batch failed (${paths.length} file(s), first: ${paths[0] ?? "?"}): ` + @@ -957,25 +1131,49 @@ async function runDaemon(): Promise { const daemon = assembleDaemon(options); - // PRD-008: attach the /api/hive-graph handlers once, after assembleDaemon, - // mirroring mountGraphApi. When Deep Lake creds resolve the group is filled, - // and the LIVE build endpoint broods (when Portkey is configured); otherwise - // the group stays scaffolded + protected but unfilled (an honest 501 scaffold). + // PRD-019b: the projects + brooding-control endpoints (consumed by Hive's + // dashboard). Persisting a toggle triggers an immediate active-set reconcile. + try { + if (activeProjects !== undefined) { + mountProjectsApi(daemon, { + view: () => { + const { resolution, cache } = readActiveProjects(controlOptions); + return buildProjectsView(resolution, cache, (id) => daemonWatcherState(daemon, id)); + }, + setProject: (projectId, brooding) => { + persistProjectBrooding(projectId, brooding, controlOptions); + }, + setGlobal: (global) => { + persistGlobalBrooding(global, controlOptions); + }, + reconcile: () => daemon.reconcileActiveProjects(), + // b-AC-6: the HTTP body carries only this path + a stable reason; the + // raw persist failure is logged server-side here. + stateFilePath: broodingStatePath(), + onPersistError: (err) => + process.stderr.write( + `nectar daemon: brooding-state persist failed: ${err instanceof Error ? err.message : String(err)}\n`, + ), + }); + } + } catch { + // fail-soft: a wiring failure never blocks the daemon from serving /health. + } + + // PRD-008: the hive-graph search/build/status API, scoped to the PRIMARY active + // project at boot (single-scope; the multi-scope dashboard surface is 019c). try { - if (ctx !== undefined) { + if (creds !== undefined && durableStore !== undefined && primary !== undefined && primaryTenancy !== undefined) { mountHiveGraphApi( daemon, buildHiveGraphApiOptions({ - credentials: ctx.credentials, - tenancy: ctx.tenancy, - projectRoot: ctx.projectRoot, - store: ctx.store, + credentials: creds, + tenancy: primaryTenancy, + projectRoot: primary.path, + store: durableStore, costSpentUsd: () => daemon.health.snapshot().cost.broodTotalUsd, - // PRD-018k / NEC-041 AC-018k.7: thread the loaded recall multiplier into - // the live search deps (the config surface reaches the recall path). ...(tunables.recallMultiplier !== undefined ? { recallMultiplier: tunables.recallMultiplier } : {}), brood: { portkey, embedProvider, metrics: daemonMetricsProxy(daemon) }, - // AC-018g.2: share the daemon's brood guard with the API /build handler. broodGuard: daemon.broodGuard, }), ); @@ -1002,8 +1200,8 @@ async function runDaemon(): Promise { // Hydrate the durable enricher's working set from Deep Lake in the BACKGROUND // (never blocks readiness; fail-soft). The enricher loop sees an empty working // set until this settles, then picks up the seeded pending rows on its next cycle. - if (live !== undefined && ctx !== undefined) { - void live.enricher.hydrate(ctx.tenancy).catch((err: unknown) => { + if (live !== undefined && primaryTenancy !== undefined) { + void live.enricher.hydrate(primaryTenancy).catch((err: unknown) => { process.stderr.write( `nectar daemon: enricher hydrate failed (non-fatal): ${err instanceof Error ? err.message : String(err)}\n`, ); @@ -1062,6 +1260,14 @@ async function main(argv: readonly string[]): Promise { return runSearchCommand(argv.slice(1)); } + if (command === "projects") { + return runProjectsCommand(argv.slice(1)); + } + + if (command === "brooding") { + return runBroodingCommand(argv.slice(1)); + } + // `project` currently exposes only its `--rebuild-projection` flag (PRD-011c); // the broader project verb surface lands with a later PRD. if (command === "project") { diff --git a/src/config-file.ts b/src/config-file.ts index 525faf1..5579611 100644 --- a/src/config-file.ts +++ b/src/config-file.ts @@ -1,5 +1,5 @@ /** - * The `~/.honeycomb/nectar.json` per-repo config-file loader (PRD-018k / NEC-041). + * The `/nectar/nectar.json` per-repo config-file loader (PRD-018k / NEC-041). * * Two knowledge docs promise a per-repo config file carrying tunables the * enricher and the recall path read: the enricher's redescribe threshold @@ -19,11 +19,11 @@ * zero runtime dependencies (AGENTS.md). */ import { existsSync, readFileSync } from "node:fs"; -import { homedir } from "node:os"; import { join } from "node:path"; -import { RUNTIME_DIR_NAME } from "./config.js"; +import { nectarStateDir } from "./apiary-root.js"; +import { resolveStateReadPath } from "./state-migration.js"; -/** The config file name within the runtime dir (`~/.honeycomb/nectar.json`). */ +/** The config file name within nectar's runtime dir (`~/.apiary/nectar/nectar.json`). */ export const NECTAR_CONFIG_FILE_NAME = "nectar.json"; /** Environment override for the enricher redescribe threshold (wins over the file). */ @@ -62,17 +62,19 @@ export interface NectarTunables { /** Options for {@link loadNectarFileConfig} / {@link resolveNectarTunables} (all injectable for tests). */ export interface NectarConfigOptions { - /** Override the runtime dir holding `nectar.json` (default: `~/.honeycomb`). */ + /** Override the runtime dir holding `nectar.json` (default: `/nectar`). */ readonly dir?: string; + /** Test seam: override the legacy runtime dir fallback path. */ + readonly legacyDir?: string; /** Env bag for the precedence layer (default: `process.env`). */ readonly env?: NodeJS.ProcessEnv; /** Warning sink (default: NDJSON to stderr). Fail-soft warnings route here. */ readonly warn?: (message: string) => void; } -/** The full `~/.honeycomb/nectar.json` path, honoring the test dir override. */ +/** The full runtime config-file path, honoring the test dir override. */ export function nectarConfigPath(options: NectarConfigOptions = {}): string { - const dir = options.dir ?? join(homedir(), RUNTIME_DIR_NAME); + const dir = options.dir ?? nectarStateDir(options.env ?? process.env); return join(dir, NECTAR_CONFIG_FILE_NAME); } @@ -86,14 +88,22 @@ function asFiniteNumber(value: unknown): number | undefined { } /** - * Load and validate `~/.honeycomb/nectar.json`. Never throws: a missing file + * Load and validate nectar's runtime `nectar.json`. Never throws: a missing file * returns `{}` silently; a malformed/non-object file or a wrong-typed known * value warns and is dropped; an unknown key warns and is skipped. Only the * two spec'd tunables are read. */ export function loadNectarFileConfig(options: NectarConfigOptions = {}): NectarFileConfig { const warn = options.warn ?? defaultWarn; - const path = nectarConfigPath(options); + const runtimeDir = options.dir ?? nectarStateDir(options.env ?? process.env); + const path = + options.dir !== undefined + ? join(runtimeDir, NECTAR_CONFIG_FILE_NAME) + : resolveStateReadPath(NECTAR_CONFIG_FILE_NAME, { + runtimeDir, + env: options.env ?? process.env, + legacyDir: options.legacyDir, + }); if (!existsSync(path)) return {}; let parsed: unknown; diff --git a/src/config.ts b/src/config.ts index 10fea9f..1c4fa86 100644 --- a/src/config.ts +++ b/src/config.ts @@ -10,9 +10,10 @@ import { homedir } from "node:os"; import { join } from "node:path"; import { ConfigError } from "./errors.js"; +import { LEGACY_RUNTIME_DIR_NAME, nectarStateDir } from "./apiary-root.js"; -/** The `~/.honeycomb` runtime dir convention shared with honeycomb + doctor. */ -export const RUNTIME_DIR_NAME = ".honeycomb"; +/** Legacy runtime dir basename retained for migration and fallback reads. */ +export const RUNTIME_DIR_NAME = LEGACY_RUNTIME_DIR_NAME; /** nectar's loopback port. 3850 honeycomb, 3851 embeddings, 3852 doctor status, 3853 hive are occupied (PRD-001b). */ export const DEFAULT_PORT = 3854; @@ -34,7 +35,7 @@ export function isLoopbackHost(host: string): boolean { /** Worker poll floor: the enricher's 30s cadence (ai/enricher-and-llm-model.md). */ export const DEFAULT_POLL_INTERVAL_MS = 30_000; -/** Distinct from honeycomb's `daemon.pid`/`daemon.lock` so both daemons coexist in ~/.honeycomb (PRD-002d). */ +/** Distinct from honeycomb's `daemon.pid`/`daemon.lock` so both daemons can coexist during migration. */ export const DEFAULT_PID_FILE_NAME = "nectar.pid"; export const DEFAULT_LOCK_FILE_NAME = "nectar.lock"; @@ -111,7 +112,7 @@ export function resolveConfig(overrides: RuntimeConfigOverrides = {}): RuntimeCo const runtimeDir = overrides.runtimeDir ?? envStr("NECTAR_RUNTIME_DIR") ?? - join(homedir(), RUNTIME_DIR_NAME); + nectarStateDir(process.env, { home: homedir() }); const host = overrides.host ?? envStr("NECTAR_HOST") ?? DEFAULT_HOST; const port = overrides.port ?? envInt("NECTAR_PORT", { min: MIN_PORT, max: MAX_PORT }) ?? DEFAULT_PORT; diff --git a/src/daemon.ts b/src/daemon.ts index b0850c4..895423a 100644 --- a/src/daemon.ts +++ b/src/daemon.ts @@ -51,14 +51,20 @@ import { import { mountHiveGraphApi, type MountHiveGraphOptions } from "./api/hive-graph-api.js"; import { createBroodGuard, type BroodGuard } from "./brood-guard.js"; import type { Timer } from "./poll-loop.js"; +import { ActiveProjectsController } from "./active-projects-runtime.js"; +import type { ProjectContextFactory } from "./project-supervisor.js"; +import type { ActiveProjectResolution } from "./hive-graph/active-projects.js"; import { createLogTap, createNullTelemetry, createTelemetry, + TELEMETRY_DB_FILE_NAME, + TELEMETRY_DIR_NAME, telemetryDbPathForRuntimeDir, type LogSink, type Telemetry, } from "./telemetry/index.js"; +import { assertNoLegacyDaemonRunning, resolveStateReadPath, runStateMigration } from "./state-migration.js"; import { HiveantennaeWorker, type JobHandler, @@ -128,6 +134,12 @@ async function raceWithTimeout(work: Promise, ms: number): Promise no + * context is constructed and `/health` reports `activeProjects: 0` with reason + * `no-active-projects` (never a brood of the cwd / `$HOME` / `System32`). + * + * Absent => the legacy single-root path stands (the explicit `projectRoot` + * override the tests and a power-user `NECTAR_PROJECT_ROOT` use). + */ + readonly activeProjects?: { + /** Resolve the current active set (reads bindings + the brooding-state file). */ + resolve(): ActiveProjectResolution; + /** Build a running brood + watch context for a project. */ + factory: ProjectContextFactory; + /** Reconcile cadence override (default: `config.pollIntervalMs`). */ + reconcileIntervalMs?: number; + /** Injected timer for the reconcile loop (deterministic tests). */ + timer?: Timer; + /** Observe a context start/stop failure. */ + onError?: (scope: "start" | "stop", projectId: string, err: unknown) => void; + }; } /** @@ -426,6 +465,12 @@ export interface AssembledDaemon { * assert liveness through `health.snapshot().watch.running` after `awaitBoot()`. */ registration(): RegistrationPipeline | null; + /** + * PRD-019a: force an immediate reconcile of the active-project set (the 019b + * toggle API calls this after persisting a brooding change). A no-op resolving + * to `undefined`-equivalent when the multi-root supervisor is not wired. + */ + reconcileActiveProjects(): Promise; /** * How many times the daemon has requested the boot cold-catch-up resync * (AC-018b.5): exactly once per successful `start()` once boot settles, and @@ -466,7 +511,8 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { health.setBroodingState({ reason: broodPrereqs.ready ? null : broodPrereqs.reason }); } - const telemetryDbPath = options.telemetryDbPath ?? telemetryDbPathForRuntimeDir(config.runtimeDir); + const runtimeDirOverride = hasExplicitRuntimeDirOverride(options); + let telemetryDbPath = options.telemetryDbPath ?? telemetryDbPathForRuntimeDir(config.runtimeDir); // A no-op placeholder until the first start() actually opens the SQLite store // (constructing/importing the daemon must stay side-effect free, unchanged). @@ -518,6 +564,21 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { // active, so at most one brood runs per daemon and the enricher never races it. const broodGuard = createBroodGuard(); + // PRD-019a: the multi-root, dormant-by-default active-project supervisor. Built + // only when the caller wires the active-projects seam (the shipped daemon does, + // via `runDaemon`); absent, the legacy single-root path below stands unchanged. + const activeProjectsController: ActiveProjectsController | null = + options.activeProjects !== undefined + ? new ActiveProjectsController({ + resolve: options.activeProjects.resolve, + factory: options.activeProjects.factory, + setHealth: (slice) => health.setActiveProjects(slice), + intervalMs: options.activeProjects.reconcileIntervalMs ?? config.pollIntervalMs, + ...(options.activeProjects.timer !== undefined ? { timer: options.activeProjects.timer } : {}), + ...(options.activeProjects.onError !== undefined ? { onError: options.activeProjects.onError } : {}), + }) + : null; + // ── PRD-016: the enricher steady-state loop ───────────────────────────────── // The loop reads/writes the SYNCHRONOUS EnricherStore seam; its per-cycle stats // feed the /health enricher slice through `enricherHealthSink`. Started on @@ -680,8 +741,11 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { log({ level: "error", scope: "registration.bridge", op, err: String(err) }); }, }); + const pendingReviewsPath = runtimeDirOverride + ? join(config.runtimeDir, "pending-reviews.json") + : resolveStateReadPath("pending-reviews.json", { runtimeDir: config.runtimeDir }); const reviews: PendingReviewStore = - options.registrationReviews ?? new FilePendingReviewStore(join(config.runtimeDir, "pending-reviews.json")); + options.registrationReviews ?? new FilePendingReviewStore(pendingReviewsPath); const registrationMetrics: PipelineMetricsSink = { incrementFilesRegistered: () => telemetry.metrics.incrementFilesRegistered(), incrementNectarsMinted: () => telemetry.metrics.incrementNectarsMinted(), @@ -772,6 +836,28 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { throw new NonLoopbackOpenApiError(config.host); } + if (!runtimeDirOverride) { + assertNoLegacyDaemonRunning({ + runtimeDir: config.runtimeDir, + pidFilePath: config.pidFilePath, + lockFilePath: config.lockFilePath, + }); + runStateMigration({ + config: { + runtimeDir: config.runtimeDir, + host: config.host, + port: config.port, + pidFilePath: config.pidFilePath, + }, + log, + }); + if (options.telemetryDbPath === undefined) { + telemetryDbPath = resolveStateReadPath(join(TELEMETRY_DIR_NAME, TELEMETRY_DB_FILE_NAME), { + runtimeDir: config.runtimeDir, + }); + } + } + // Step 5 (PRD-002a): acquire the single-instance lock BEFORE the bind, and // remember the identity we stamped so the rollback path releases only what // THIS instance acquired (PRD-018a NEC-002). @@ -857,6 +943,18 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { await startRegistrationPipeline(); })(); + // PRD-019a: publish the current active-project resolution to /health + // immediately (so a dormant daemon reads activeProjects:0 with reason + // no-active-projects right away), then arm the reconcile loop. + if (activeProjectsController !== null) { + try { + activeProjectsController.publishHealth(); + } catch (err) { + log({ level: "error", scope: "active-projects", msg: "initial publish failed", err: String(err) }); + } + activeProjectsController.start(); + } + return boundPort; })(); @@ -898,6 +996,15 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { // sees `closed` and never starts the watcher during teardown. registration?.service.stop(); if (registration !== null) health.setWatchState({ running: false }); + // PRD-019a: stop the reconcile loop and tear down every running project + // context (watcher stopped + bridge writes drained) before the lock releases. + if (activeProjectsController !== null) { + try { + await activeProjectsController.stop(); + } catch (err) { + log({ level: "error", scope: "active-projects", msg: "stop failed", err: String(err) }); + } + } // AC-018a.10/11 (NEC-033): await the in-flight worker tick and bootSettled // under a bounded timeout so a shutdown that catches the worker busy drains @@ -965,6 +1072,10 @@ export function assembleDaemon(options: AssembleOptions = {}): AssembledDaemon { awaitBoot: () => bootSettled, group: (path) => router.group(path), registration: () => registration, + reconcileActiveProjects: async () => { + if (activeProjectsController === null) return; + await activeProjectsController.reconcileNow(); + }, registrationBootResyncCount: () => bootResyncCount, }; diff --git a/src/doctor-registry.ts b/src/doctor-registry.ts index 2e755f7..0f4952e 100644 --- a/src/doctor-registry.ts +++ b/src/doctor-registry.ts @@ -2,7 +2,8 @@ * nectar's registry entry in doctor's daemon registry (PRD-003c). * * nectar becomes a supervised daemon by appending ONE entry to doctor's - * registry file (`~/.honeycomb/doctor.daemons.json`, the schema PRD-004a + * registry file (`/registry.json` when the fleet root exists, + * else legacy `~/.honeycomb/doctor.daemons.json`, the schema PRD-004a * specifies, mirrored read-side in `doctor/src/registry.ts`). That entry * carries nectar's `healthUrl`, `pidPath`, `probeIntervalMs`, `startupGraceMs`, * and restart thresholds. At the next doctor boot, doctor reads the @@ -31,7 +32,7 @@ * * PRD-017a extends this entry with `telemetryDbPath`: the absolute path to * nectar's runtime telemetry SQLite database (`telemetry/db.ts`, - * `~/.honeycomb/telemetry/nectar.sqlite` by default), per doctor's + * `~/.apiary/nectar/telemetry/nectar.sqlite` by default), per doctor's * `ADR-0002-service-registration-static-registry-plus-runtime-sqlite.md`. It is * derived from `config.pidFilePath`'s directory (the same resolved runtime dir * `healthUrl`/`pidPath` already come from), so a test-overridden runtime dir @@ -42,9 +43,10 @@ * Built-ins only: node:fs, node:os, node:path. */ -import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs"; +import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs"; import { homedir } from "node:os"; import { basename, dirname, join } from "node:path"; +import { legacyRuntimeDir, resolveApiaryRoot } from "./apiary-root.js"; import type { RuntimeConfig } from "./config.js"; import { TELEMETRY_DB_FILE_NAME, TELEMETRY_DIR_NAME } from "./telemetry/db.js"; @@ -106,9 +108,11 @@ export class DoctorRegistryError extends Error { } } -/** The default registry file location, alongside the other `~/.honeycomb` artifacts. */ -export function defaultDoctorRegistryPath(home: string = homedir()): string { - return join(home, ".honeycomb", "doctor.daemons.json"); +/** The default registry file location per the ADR compatibility-window contract. */ +export function defaultDoctorRegistryPath(home: string = homedir(), env: NodeJS.ProcessEnv = process.env): string { + const fleetRoot = resolveApiaryRoot(env, { home }); + if (existsSync(fleetRoot)) return join(fleetRoot, "registry.json"); + return join(legacyRuntimeDir(home), "doctor.daemons.json"); } /** diff --git a/src/health.ts b/src/health.ts index c710ac9..29dca0e 100644 --- a/src/health.ts +++ b/src/health.ts @@ -47,6 +47,27 @@ export interface HealthBody { portkey: { enabled: boolean; }; + /** + * PRD-019a: the active-project set the daemon broods + watches. A daemon with + * no bound, brooding-enabled projects is DORMANT: `count` is 0 and `reason` is + * `"no-active-projects"` (never a brood of the cwd / `$HOME` / `System32`). + * Each entry carries the resolved tenancy project id, the bound path, its + * effective brooding state, and its watcher liveness. `refused` lists bound + * roots that resolved to a guarded path (`$HOME` / filesystem root / System32) + * with reason `"pathological-root"`. + */ + activeProjects: { + count: number; + /** `"no-active-projects"` when zero are active, `"global-paused"` when the global switch is paused, else null. */ + reason: string | null; + projects: Array<{ + projectId: string; + path: string; + brooding: "active" | "paused" | "global-paused"; + watcher: "stopped" | "running" | "restarting" | "degraded"; + }>; + refused: Array<{ projectId: string; path: string; reason: string }>; + }; watch: { /** True once the update-on-change registration pipeline's NodeFS watcher is running (PRD-018b). */ running: boolean; @@ -107,6 +128,12 @@ export class HealthState { }; readonly embeddings: HealthBody["embeddings"] = { provider: "off" }; readonly portkey: HealthBody["portkey"] = { enabled: false }; + readonly activeProjects: HealthBody["activeProjects"] = { + count: 0, + reason: "no-active-projects", + projects: [], + refused: [], + }; readonly watch: HealthBody["watch"] = { running: false, reason: null, @@ -202,6 +229,18 @@ export class HealthState { this.watch.lastFlushErrorAt = atIso; } + /** + * Replace the active-project slice (PRD-019a). The reconcile driver calls this + * after every reconcile cycle with the resolved set + per-project watcher + * liveness, so `/health` reflects exactly what the daemon broods and watches. + */ + setActiveProjects(state: HealthBody["activeProjects"]): void { + this.activeProjects.count = state.count; + this.activeProjects.reason = state.reason; + this.activeProjects.projects = state.projects.map((p) => ({ ...p })); + this.activeProjects.refused = state.refused.map((r) => ({ ...r })); + } + setStatus(status: PipelineStatus): void { this.status = status; } @@ -225,6 +264,12 @@ export class HealthState { cost: { ...this.cost }, embeddings: { ...this.embeddings }, portkey: { ...this.portkey }, + activeProjects: { + count: this.activeProjects.count, + reason: this.activeProjects.reason, + projects: this.activeProjects.projects.map((p) => ({ ...p })), + refused: this.activeProjects.refused.map((r) => ({ ...r })), + }, watch: { ...this.watch }, }; } diff --git a/src/hive-graph/active-projects.ts b/src/hive-graph/active-projects.ts new file mode 100644 index 0000000..0e9a76d --- /dev/null +++ b/src/hive-graph/active-projects.ts @@ -0,0 +1,198 @@ +/** + * Active-project resolution for the dormant-by-default multi-root daemon (PRD-019a). + * + * The daemon's brood + watch scope is the set of folder bindings read from the + * shared `~/.deeplake/projects.json` (via `loadProjectsCache`) whose per-project + * brooding flag is ON and whose global switch is not paused (PRD-019b), MINUS + * any binding whose path resolves to a guarded ("pathological") root. With zero + * such bindings the daemon is dormant: it broods nothing, watches nothing, and + * NEVER falls back to `process.cwd()`, `$HOME`, `/`, or `System32`. + * + * Pure functions of their inputs (bindings snapshot, brooding state, home / + * platform / env), so the whole resolution is unit-testable without disk. + */ +import { homedir } from "node:os"; +import { posix, win32 } from "node:path"; +import type { FolderBinding } from "./project-scope.js"; +import { effectiveBrooding, type BroodingState, type EffectiveBrooding } from "../registration/brooding-state.js"; + +/** A bound project resolved into the active-set view (with its effective brooding). */ +export interface ResolvedProject { + readonly projectId: string; + readonly path: string; + readonly brooding: EffectiveBrooding; +} + +/** A bound root refused activation because it resolved to a guarded path. */ +export interface RefusedProject { + readonly projectId: string; + readonly path: string; + readonly reason: "pathological-root"; +} + +/** The full resolution: every bound project (with brooding state), the refused set, and the active subset. */ +export interface ActiveProjectResolution { + /** Every bound, non-refused project with its effective brooding state (for `/health`). */ + readonly projects: readonly ResolvedProject[]; + /** Bound roots refused for resolving to a guarded path. */ + readonly refused: readonly RefusedProject[]; + /** The subset of `projects` whose brooding is `"active"` - the daemon's brood + watch targets. */ + readonly active: readonly ResolvedProject[]; + /** True when the global switch is paused (so nothing is active regardless of per-project state). */ + readonly globalPaused: boolean; +} + +export interface PathologicalRootOptions { + /** The user's home directory (default: `os.homedir()`). Injectable for tests. */ + readonly home?: string; + /** The platform whose root/System32 conventions apply (default: `process.platform`). */ + readonly platform?: NodeJS.Platform; + /** Env bag (for `%WINDIR%`; default: `process.env`). */ + readonly env?: NodeJS.ProcessEnv; +} + +/** + * The path module for the TARGET platform's conventions. The guard is a pure + * function of `(path, platform)`, so a win32-shaped fixture (`C:\`) evaluated + * in a test running on POSIX must still use win32 semantics, and vice versa, + * instead of the HOST's `node:path` default (which made the guard's verdicts + * host-dependent: `path.parse("C:\\").root` is `""` on POSIX, so a drive root + * was not recognized as a filesystem root when the suite ran on macOS/Linux). + */ +function pathModuleFor(platform: NodeJS.Platform): typeof posix | typeof win32 { + return platform === "win32" ? win32 : posix; +} + +/** + * Normalize a path for comparison under the target platform's conventions: + * forward-slashed, no trailing separator, and case-folded ONLY on win32 + * (POSIX is case-sensitive; `$HOME` must not be compared case-insensitively + * there). Backslashes are treated as separators only for win32 shapes; on + * POSIX a backslash is a legal filename character. + */ +function normalizeForCompare(p: string, platform: NodeJS.Platform): string { + const pathMod = pathModuleFor(platform); + const withForwardSlashes = platform === "win32" ? p.replace(/\\/g, "/") : p; + const normalized = pathMod.normalize(withForwardSlashes).replace(/\\/g, "/").replace(/\/+$/, ""); + const trimmed = normalized === "" ? "/" : normalized; + return platform === "win32" ? trimmed.toLowerCase() : trimmed; +} + +/** + * True when `rootPath` resolves to a guarded root that must never be brooded, + * even when explicitly bound (defense in depth against a mis-bind, PRD-019a): + * the user's `$HOME`, a filesystem/drive root (`/`, `C:\`), or, on Windows + * only, `%WINDIR%\System32`. Evaluated under the TARGET platform's path + * conventions (injectable for tests), so verdicts do not vary by test host. + */ +export function isPathologicalRoot(rootPath: string, options: PathologicalRootOptions = {}): boolean { + const platform = options.platform ?? process.platform; + const env = options.env ?? process.env; + const pathMod = pathModuleFor(platform); + // Resolve relative inputs against the HOST cwd (the only meaningful anchor), + // but parse/compare under the TARGET conventions. + const resolved = pathMod.isAbsolute(rootPath) ? rootPath : pathMod.resolve(rootPath); + const norm = normalizeForCompare(resolved, platform); + + // Filesystem / drive root: the parsed root equals the whole path (`/`, + // `C:\`, `//server/share` degenerate forms). Parsed under the TARGET + // platform so `C:\` is a drive root even when the test host is POSIX. + const parsedRoot = pathMod.parse(resolved).root; + if (parsedRoot !== "" && norm === normalizeForCompare(parsedRoot, platform)) return true; + // Belt-and-suspenders for an empty/`/` normalization result. + if (norm === "/" || norm === "") return true; + + // $HOME (case-sensitive comparison on POSIX; folded only on win32). + const home = options.home ?? homedir(); + if (home !== undefined && home.trim() !== "" && norm === normalizeForCompare(home, platform)) return true; + + // %WINDIR%\System32 (the Windows Scheduled Task default cwd). WINDOWS-ONLY: + // a POSIX daemon must never false-positive on a directory that merely spells + // /Windows/System32, and env.WINDIR is meaningless off-Windows. + if (platform === "win32") { + const windir = env.WINDIR ?? env.windir; + const systemRoot = windir !== undefined && windir.trim() !== "" ? windir : "C:/Windows"; + const sys32 = normalizeForCompare(`${systemRoot.replace(/\\/g, "/")}/System32`, platform); + if (norm === sys32) return true; + } + + return false; +} + +export interface ResolveActiveProjectsInput { + /** The folder bindings from the shared `~/.deeplake/projects.json` (via `loadProjectsCache`). */ + readonly bindings: readonly FolderBinding[]; + /** The nectar-owned brooding state (per-project + global switch). */ + readonly broodingState: BroodingState; + /** Home directory for the pathological-root guard (default: `os.homedir()`). */ + readonly home?: string; + /** Platform for the pathological-root guard (default: `process.platform`). */ + readonly platform?: NodeJS.Platform; + /** Env bag for the pathological-root guard (default: `process.env`). */ + readonly env?: NodeJS.ProcessEnv; +} + +/** + * Resolve the active-project set from the folder bindings + brooding state. + * De-duplicates by project id (first binding wins). A binding whose path is a + * guarded root is refused, not activated. Returns every bound (non-refused) + * project with its effective brooding for `/health`, plus the `active` subset + * (brooding `"active"`) the supervisor stands contexts up for. + */ +export function resolveActiveProjects(input: ResolveActiveProjectsInput): ActiveProjectResolution { + const guardOptions: PathologicalRootOptions = { + ...(input.home !== undefined ? { home: input.home } : {}), + ...(input.platform !== undefined ? { platform: input.platform } : {}), + ...(input.env !== undefined ? { env: input.env } : {}), + }; + + const projects: ResolvedProject[] = []; + const refused: RefusedProject[] = []; + const seen = new Set(); + + for (const binding of input.bindings) { + const projectId = binding.projectId.trim(); + if (projectId === "" || binding.path.trim() === "") continue; + if (seen.has(projectId)) continue; + seen.add(projectId); + + if (isPathologicalRoot(binding.path, guardOptions)) { + refused.push({ projectId, path: binding.path, reason: "pathological-root" }); + continue; + } + projects.push({ projectId, path: binding.path, brooding: effectiveBrooding(input.broodingState, projectId) }); + } + + const active = projects.filter((p) => p.brooding === "active"); + return { + projects, + refused, + active, + globalPaused: input.broodingState.globalBrooding === "paused", + }; +} + +/** Build the `/health` `activeProjects` slice from a resolution + a per-project watcher-state lookup. */ +export function activeProjectsHealth( + resolution: ActiveProjectResolution, + watcherStateFor: (projectId: string) => "stopped" | "running" | "restarting" | "degraded", +): { + count: number; + reason: string | null; + projects: Array<{ projectId: string; path: string; brooding: "active" | "paused" | "global-paused"; watcher: "stopped" | "running" | "restarting" | "degraded" }>; + refused: Array<{ projectId: string; path: string; reason: string }>; +} { + const count = resolution.active.length; + const reason = count > 0 ? null : resolution.globalPaused ? "global-paused" : "no-active-projects"; + return { + count, + reason, + projects: resolution.projects.map((p) => ({ + projectId: p.projectId, + path: p.path, + brooding: p.brooding, + watcher: p.brooding === "active" ? watcherStateFor(p.projectId) : "stopped", + })), + refused: resolution.refused.map((r) => ({ projectId: r.projectId, path: r.path, reason: r.reason })), + }; +} diff --git a/src/index.ts b/src/index.ts index a258fd7..8910c60 100644 --- a/src/index.ts +++ b/src/index.ts @@ -8,6 +8,14 @@ */ export { assembleDaemon } from "./daemon.js"; export type { AssembleOptions, AssembledDaemon, RegistrationPipeline } from "./daemon.js"; +export { + resolveApiaryRoot, + nectarStateDir, + legacyRuntimeDir, + APIARY_ROOT_DIR_NAME, + NECTAR_STATE_DIR_NAME, + LEGACY_RUNTIME_DIR_NAME, +} from "./apiary-root.js"; export { resolveConfig, DEFAULT_HOST, diff --git a/src/project-supervisor.ts b/src/project-supervisor.ts new file mode 100644 index 0000000..c751700 --- /dev/null +++ b/src/project-supervisor.ts @@ -0,0 +1,129 @@ +/** + * The multi-root project supervisor (PRD-019a). + * + * Owns a `Map` and reconciles it against the active + * set on demand (the daemon's poll cadence + the 019b toggle API). A newly + * active project is started (hydrate mirror, start watcher, request cold-catch-up + * resync, arm auto-brood); a newly inactive one is stopped and drained; the + * daemon never restarts. Reconciles are serialized so two overlapping cycles + * never double-start a context. + * + * The supervisor is deliberately generic: the per-project `RunningContext` and + * how it is built are injected via a {@link ProjectContextFactory}, so the + * reconcile logic is unit-testable with fakes and the live daemon wires the real + * registration + brood context. + */ +import type { WatcherState } from "./registration/fs-watch.js"; +import type { ResolvedProject } from "./hive-graph/active-projects.js"; + +/** One project's running brood + watch context. `start`/`stop` are idempotent from the supervisor's view. */ +export interface RunningContext { + readonly projectId: string; + readonly path: string; + /** The current watcher liveness for the `/health` slice. */ + watcherState(): WatcherState; + /** Hydrate the mirror, start the watcher, request the cold-catch-up resync, arm auto-brood. */ + start(): Promise; + /** Stop the watcher and drain bridge writes before the context is released (a-AC-7). */ + stop(): Promise; +} + +/** Build a {@link RunningContext} for a project (does NOT start it; the supervisor calls `start`). */ +export type ProjectContextFactory = (project: ResolvedProject) => RunningContext; + +export interface ProjectSupervisorOptions { + readonly factory: ProjectContextFactory; + /** Observe a context start/stop failure (never throws out of reconcile). Default: no-op. */ + readonly onError?: (scope: "start" | "stop", projectId: string, err: unknown) => void; +} + +/** The outcome of one reconcile pass (for logging/tests). */ +export interface ReconcileOutcome { + readonly started: readonly string[]; + readonly stopped: readonly string[]; +} + +export class ProjectSupervisor { + private readonly factory: ProjectContextFactory; + private readonly onError: (scope: "start" | "stop", projectId: string, err: unknown) => void; + private readonly contextsById = new Map(); + /** Serializes reconciles so two overlapping cycles never double-start a context. */ + private queue: Promise = Promise.resolve({ started: [], stopped: [] }); + + constructor(options: ProjectSupervisorOptions) { + this.factory = options.factory; + this.onError = options.onError ?? (() => {}); + } + + /** The currently running contexts (snapshot). */ + contexts(): readonly RunningContext[] { + return [...this.contextsById.values()]; + } + + /** The running context for a project id, or undefined. */ + get(projectId: string): RunningContext | undefined { + return this.contextsById.get(projectId); + } + + /** The watcher state for a running project, or `"stopped"` when it is not running. */ + watcherStateFor(projectId: string): WatcherState { + const ctx = this.contextsById.get(projectId); + if (ctx === undefined) return "stopped"; + try { + return ctx.watcherState(); + } catch { + return "stopped"; + } + } + + /** + * Reconcile the running contexts against `active`: start any project not + * running (or whose bound path changed), stop any running project no longer + * active. Serialized behind the internal queue; never throws (start/stop + * failures route to `onError`). + */ + reconcile(active: readonly ResolvedProject[]): Promise { + this.queue = this.queue.then(() => this.reconcileNow(active)); + return this.queue; + } + + private async reconcileNow(active: readonly ResolvedProject[]): Promise { + const started: string[] = []; + const stopped: string[] = []; + const activeById = new Map(active.map((p) => [p.projectId, p] as const)); + + // Stop contexts that are no longer active, or whose bound path changed. + for (const [projectId, ctx] of [...this.contextsById.entries()]) { + const target = activeById.get(projectId); + if (target === undefined || target.path !== ctx.path) { + this.contextsById.delete(projectId); + try { + await ctx.stop(); + } catch (err) { + this.onError("stop", projectId, err); + } + stopped.push(projectId); + } + } + + // Start contexts that are newly active (or were just stopped for a path change). + for (const project of active) { + if (this.contextsById.has(project.projectId)) continue; + const ctx = this.factory(project); + this.contextsById.set(project.projectId, ctx); + try { + await ctx.start(); + } catch (err) { + this.onError("start", project.projectId, err); + } + started.push(project.projectId); + } + + return { started, stopped }; + } + + /** Stop every running context and clear the map (shutdown / teardown, a-AC-7). Serialized behind the queue. */ + stopAll(): Promise { + return this.reconcile([]); + } +} diff --git a/src/projects-control.ts b/src/projects-control.ts new file mode 100644 index 0000000..ffcc849 --- /dev/null +++ b/src/projects-control.ts @@ -0,0 +1,130 @@ +/** + * The reusable projects + brooding-control core (PRD-019b). + * + * One place that composes the shared `~/.deeplake/projects.json` bindings, the + * nectar-owned brooding state, and the active-set resolution into the read view + * the `GET /api/hive-graph/projects` endpoint and the `nectar projects` CLI both + * render, plus the persist helpers the `POST .../projects/brooding` endpoint and + * the `nectar brooding` CLI both call. Pure of HTTP/CLI concerns so both surfaces + * produce the identical persisted + reconciled effect (b-AC-7). + */ +import { loadProjectsCache, type ProjectsCache } from "./hive-graph/project-scope.js"; +import { + loadBroodingState, + writeBroodingState, + withGlobalBrooding, + withProjectBrooding, + type BroodingState, + type BroodingStateOptions, + type GlobalBrooding, + type ProjectBrooding, +} from "./registration/brooding-state.js"; +import { resolveActiveProjects, type ActiveProjectResolution } from "./hive-graph/active-projects.js"; +import type { WatcherState } from "./registration/fs-watch.js"; + +/** The global switch value as surfaced to the dashboard. */ +export type GlobalBroodingView = GlobalBrooding; + +/** One project's row in the read view (PRD-019b GET /projects shape). */ +export interface ProjectBroodingEntry { + readonly projectId: string; + readonly name: string; + readonly path: string; + readonly brooding: "active" | "paused" | "global-paused"; + readonly watcher: WatcherState; + /** + * Per-project brood/enrich counts. Reserved for a later per-project counts + * surface; the current daemon tracks brood counts globally, so this is `null` + * (honest) rather than a fabricated per-project number. + */ + readonly counts: null; +} + +/** The `GET /api/hive-graph/projects` (and `nectar projects`) read view. */ +export interface ProjectsView { + readonly globalBrooding: GlobalBroodingView; + readonly projects: readonly ProjectBroodingEntry[]; +} + +export interface ProjectsControlOptions { + /** Override the `~/.deeplake` cache dir (tests). */ + readonly cacheDir?: string; + /** Tenancy guard for the shared cache (org/workspace the daemon authenticated as). */ + readonly expect?: { org: string; workspace: string }; + /** Override the brooding-state file location (tests). */ + readonly broodingState?: BroodingStateOptions; + /** Home for the pathological-root guard (default: `os.homedir()`). */ + readonly home?: string; + /** Platform for the pathological-root guard (default: `process.platform`). */ + readonly platform?: NodeJS.Platform; + /** Env for the pathological-root guard (default: `process.env`). */ + readonly env?: NodeJS.ProcessEnv; +} + +/** Load the bindings + brooding state and resolve the active set. */ +export function readActiveProjects(options: ProjectsControlOptions = {}): { + readonly resolution: ActiveProjectResolution; + readonly cache: ProjectsCache; + readonly state: BroodingState; +} { + const cache = loadProjectsCache({ + ...(options.cacheDir !== undefined ? { dir: options.cacheDir } : {}), + ...(options.expect !== undefined ? { expect: options.expect } : {}), + }); + const state = loadBroodingState(options.broodingState ?? {}); + const resolution = resolveActiveProjects({ + bindings: cache.bindings, + broodingState: state, + ...(options.home !== undefined ? { home: options.home } : {}), + ...(options.platform !== undefined ? { platform: options.platform } : {}), + ...(options.env !== undefined ? { env: options.env } : {}), + }); + return { resolution, cache, state }; +} + +/** Build the read view from a resolution + cache + a per-project watcher-state lookup. */ +export function buildProjectsView( + resolution: ActiveProjectResolution, + cache: ProjectsCache, + watcherStateFor: (projectId: string) => WatcherState, +): ProjectsView { + const nameById = new Map(cache.projects.map((p) => [p.projectId, p.name] as const)); + return { + globalBrooding: resolution.globalPaused ? "paused" : "on", + projects: resolution.projects.map((p) => ({ + projectId: p.projectId, + name: nameById.get(p.projectId) ?? "", + path: p.path, + brooding: p.brooding, + watcher: p.brooding === "active" ? watcherStateFor(p.projectId) : "stopped", + counts: null, + })), + }; +} + +/** + * Persist a per-project brooding flag (atomic write). Throws on a write failure + * so the caller preserves the prior state and skips the reconcile (b-AC-6). + * Returns the new state. + */ +export function persistProjectBrooding( + projectId: string, + brooding: ProjectBrooding, + options: ProjectsControlOptions = {}, +): BroodingState { + const state = loadBroodingState(options.broodingState ?? {}); + const next = withProjectBrooding(state, projectId, brooding); + writeBroodingState(next, options.broodingState ?? {}); + return next; +} + +/** Persist the global switch (atomic write). Throws on a write failure (b-AC-6). Returns the new state. */ +export function persistGlobalBrooding( + global: GlobalBrooding, + options: ProjectsControlOptions = {}, +): BroodingState { + const state = loadBroodingState(options.broodingState ?? {}); + const next = withGlobalBrooding(state, global); + writeBroodingState(next, options.broodingState ?? {}); + return next; +} diff --git a/src/registration/brooding-state.ts b/src/registration/brooding-state.ts new file mode 100644 index 0000000..d821176 --- /dev/null +++ b/src/registration/brooding-state.ts @@ -0,0 +1,197 @@ +/** + * The nectar-owned brooding-state store (PRD-019b). + * + * Records per-project and global brooding on/off in a nectar-owned JSON file at + * `/nectar/projects.json` (fleet ADR-0003 / nectar ADR-0005). This + * is NOT the shared `~/.deeplake/projects.json` (which carries the folder + * bindings nectar reads for scope, per ADR-0002); nectar's own control state + * lives under its own per-product subdirectory of the neutral fleet root, + * created on first write, so it never depends on honeycomb being installed. + * + * Fail-soft loader/writer built on `node:fs` only (zero runtime dependencies, + * mirroring `src/config-file.ts`): a missing file, malformed JSON, or a + * non-object payload reads as defaults (global `on`, each project defaulting to + * `on` when first seen); unknown keys warn and are skipped (forward + * compatibility); writes are atomic (temp file + rename) and create the + * directory on first write. + */ +import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { nectarStateDir } from "../apiary-root.js"; + +/** The nectar-owned brooding-state file name (under `/nectar/`). */ +export const BROODING_STATE_FILE_NAME = "projects.json"; + +/** The schema version this reader/writer understands. */ +export const BROODING_STATE_SCHEMA_VERSION = 1 as const; + +/** The global brooding switch value. `paused` is the runtime-toggleable emergency pause. */ +export type GlobalBrooding = "on" | "paused"; + +/** A per-project brooding value. */ +export type ProjectBrooding = "on" | "off"; + +/** The default a newly-seen bound project takes (PRD-019 index resolved decision: ON). */ +export const DEFAULT_PROJECT_BROODING: ProjectBrooding = "on"; + +/** The default global switch value (PRD-019 index resolved decision: ON). */ +export const DEFAULT_GLOBAL_BROODING: GlobalBrooding = "on"; + +/** + * The effective brooding state for a project after AND-ing the global switch with + * the per-project flag: the global pause always wins, then a per-project `off` + * pauses, else the project is active. + */ +export type EffectiveBrooding = "active" | "paused" | "global-paused"; + +/** The validated brooding state (defaults when the file is missing/malformed). */ +export interface BroodingState { + readonly schemaVersion: typeof BROODING_STATE_SCHEMA_VERSION; + readonly globalBrooding: GlobalBrooding; + /** projectId -> per-project brooding flag. A project absent here defaults to {@link DEFAULT_PROJECT_BROODING}. */ + readonly projects: Readonly>; +} + +export interface BroodingStateOptions { + /** Override the directory holding `projects.json` (default: `/nectar`). */ + readonly dir?: string; + /** Env bag for resolving the fleet root (default: `process.env`). */ + readonly env?: NodeJS.ProcessEnv; + /** Warning sink (default: NDJSON to stderr). Fail-soft warnings route here. */ + readonly warn?: (message: string) => void; +} + +/** Resolve the directory holding the brooding-state file (honors the test override). */ +export function broodingStateDir(options: BroodingStateOptions = {}): string { + return options.dir ?? nectarStateDir(options.env ?? process.env); +} + +/** Resolve the full brooding-state file path (honors the test override). */ +export function broodingStatePath(options: BroodingStateOptions = {}): string { + return join(broodingStateDir(options), BROODING_STATE_FILE_NAME); +} + +/** The defaults a missing/malformed file falls soft to. */ +export function defaultBroodingState(): BroodingState { + return { schemaVersion: BROODING_STATE_SCHEMA_VERSION, globalBrooding: DEFAULT_GLOBAL_BROODING, projects: {} }; +} + +function defaultWarn(message: string): void { + process.stderr.write( + `${JSON.stringify({ ts: new Date().toISOString(), level: "warn", scope: "brooding-state", msg: message })}\n`, + ); +} + +function isGlobalBrooding(v: unknown): v is GlobalBrooding { + return v === "on" || v === "paused"; +} + +function isProjectBrooding(v: unknown): v is ProjectBrooding { + return v === "on" || v === "off"; +} + +/** + * Load and validate the brooding-state file. Never throws: a missing file + * returns the defaults silently; malformed JSON, a non-object payload, or a + * wrong `schemaVersion` warns and returns the defaults; unknown top-level keys + * and malformed per-project values warn and are skipped (forward compatibility). + */ +export function loadBroodingState(options: BroodingStateOptions = {}): BroodingState { + const warn = options.warn ?? defaultWarn; + const path = broodingStatePath(options); + if (!existsSync(path)) return defaultBroodingState(); + + let parsed: unknown; + try { + parsed = JSON.parse(readFileSync(path, "utf8")); + } catch { + warn(`ignoring malformed ${path} (not valid JSON); falling back to defaults`); + return defaultBroodingState(); + } + if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) { + warn(`ignoring ${path} (not a JSON object); falling back to defaults`); + return defaultBroodingState(); + } + + const record = parsed as Record; + if (record.schemaVersion !== undefined && record.schemaVersion !== BROODING_STATE_SCHEMA_VERSION) { + warn(`ignoring ${path} (unsupported schemaVersion ${JSON.stringify(record.schemaVersion)}); falling back to defaults`); + return defaultBroodingState(); + } + + let globalBrooding: GlobalBrooding = DEFAULT_GLOBAL_BROODING; + const projects: Record = {}; + + for (const key of Object.keys(record)) { + if (key === "schemaVersion") continue; + if (key === "globalBrooding") { + if (isGlobalBrooding(record[key])) globalBrooding = record[key] as GlobalBrooding; + else warn(`ignoring invalid globalBrooding ${JSON.stringify(record[key])} in ${path}`); + continue; + } + if (key === "projects") { + const raw = record[key]; + if (typeof raw !== "object" || raw === null || Array.isArray(raw)) { + warn(`ignoring invalid projects map in ${path}`); + continue; + } + for (const [projectId, value] of Object.entries(raw as Record)) { + if (typeof value === "object" && value !== null && !Array.isArray(value)) { + const brooding = (value as Record).brooding; + if (isProjectBrooding(brooding)) projects[projectId] = brooding; + else warn(`ignoring invalid brooding for project ${JSON.stringify(projectId)} in ${path}`); + } else { + warn(`ignoring malformed entry for project ${JSON.stringify(projectId)} in ${path}`); + } + } + continue; + } + warn(`ignoring unknown key ${JSON.stringify(key)} in ${path}`); + } + + return { schemaVersion: BROODING_STATE_SCHEMA_VERSION, globalBrooding, projects }; +} + +/** Serialize the state to the on-disk JSON shape (`{ schemaVersion, globalBrooding, projects: { id: { brooding } } }`). */ +function serialize(state: BroodingState): string { + const projects: Record = {}; + for (const [projectId, brooding] of Object.entries(state.projects)) { + projects[projectId] = { brooding }; + } + return `${JSON.stringify({ schemaVersion: BROODING_STATE_SCHEMA_VERSION, globalBrooding: state.globalBrooding, projects }, null, 2)}\n`; +} + +/** + * Atomically write the brooding state (temp file + rename), creating the + * directory on first write. Mirrors the projection/registry write discipline. + * Throws on a hard IO failure so the caller can preserve the prior state and + * skip the reconcile (b-AC-6). + */ +export function writeBroodingState(state: BroodingState, options: BroodingStateOptions = {}): void { + const path = broodingStatePath(options); + const dir = dirname(path); + // Create the nectar state dir owner-only (0o700), matching the other state + // writers (state-migration `copyThenRename`, telemetry-usage `saveLedger`); + // a default-mode mkdir would leave the runtime state dir group/other-traversable. + mkdirSync(dir, { recursive: true, mode: 0o700 }); + const tmpPath = join(dir, `.${BROODING_STATE_FILE_NAME}.${process.pid}.${Date.now()}.tmp`); + writeFileSync(tmpPath, serialize(state), "utf8"); + renameSync(tmpPath, path); +} + +/** Resolve the effective brooding for a project id (global pause beats per-project off). */ +export function effectiveBrooding(state: BroodingState, projectId: string): EffectiveBrooding { + if (state.globalBrooding === "paused") return "global-paused"; + const project = state.projects[projectId] ?? DEFAULT_PROJECT_BROODING; + return project === "off" ? "paused" : "active"; +} + +/** Return a new state with `projectId`'s brooding set to `brooding` (immutable update). */ +export function withProjectBrooding(state: BroodingState, projectId: string, brooding: ProjectBrooding): BroodingState { + return { ...state, projects: { ...state.projects, [projectId]: brooding } }; +} + +/** Return a new state with the global switch set to `global` (immutable update). */ +export function withGlobalBrooding(state: BroodingState, global: GlobalBrooding): BroodingState { + return { ...state, globalBrooding: global }; +} diff --git a/src/registration/disk-fs.ts b/src/registration/disk-fs.ts index b4204bc..77eb5f8 100644 --- a/src/registration/disk-fs.ts +++ b/src/registration/disk-fs.ts @@ -16,7 +16,7 @@ import { lstatSync, readdirSync, readFileSync, statSync, unlinkSync, writeFileSync, type Dirent } from "node:fs"; import { join, relative } from "node:path"; import { containedPath, isSafeRelPath, realpathContained } from "./paths-safe.js"; -import type { IgnorePredicate } from "./ignore.js"; +import { createDefaultIgnore, type IgnorePredicate } from "./ignore.js"; import type { RegistrationFs, StatResult } from "./service.js"; /** @@ -45,7 +45,11 @@ const defaultReadDirSync: ReadDirSync = (dir) => readdirSync(dir, { withFileType export function createDiskRegistrationFs( root: string, - isIgnored: IgnorePredicate = () => false, + // PRD-019d / d-AC-7: an omitted predicate still drops the always-ignored + // segments (`.git` / `node_modules` / `.honeycomb`) plus any `graph-ignore.json` + // prefixes, rather than ignoring nothing - defense in depth against a caller + // that forgets to pass the shared predicate. + isIgnored: IgnorePredicate = createDefaultIgnore(root), readDirSync: ReadDirSync = defaultReadDirSync, ): RegistrationFs { return { diff --git a/src/registration/gitignore.ts b/src/registration/gitignore.ts new file mode 100644 index 0000000..8c465cf --- /dev/null +++ b/src/registration/gitignore.ts @@ -0,0 +1,282 @@ +/** + * A dependency-free `.gitignore` matcher for the walk fallback (PRD-019d). + * + * PRD-018c's shared ignore predicate approximates gitignore semantics with a + * cached `git ls-files` snapshot. When git is genuinely absent (no `.git`, no + * `git` on PATH), that snapshot is unavailable and the pre-019d predicate read + * "nothing ignored" for the gitignore layer, so a bound non-git subtree ingested + * everything but the always-ignored segments. This module fills exactly that + * gap: a pure gitignore matcher plus a thin disk loader, used ONLY when the git + * snapshot cannot answer. When git IS available the `ls-files` snapshot stays + * authoritative (it already reflects `.gitignore`), so the parser is a fallback, + * never a second opinion that could disagree. + * + * Node built-ins only (AGENTS.md: zero runtime dependencies). Supported: `!` + * negation, trailing-`/` directory-only patterns, leading-`/` (or embedded-`/`) + * anchoring, `**`, `*`, `?`, and bare basename globs. Nested `.gitignore` files + * and `.git/info/exclude` are read by the disk loader with correct precedence + * (deeper files override shallower; `.git/info/exclude` is lowest). Conservative + * by design: when in doubt the matcher does NOT ignore (fail-open to inclusion is + * safer for a memory layer than dropping a file the user expected indexed). Known + * out of scope: `core.excludesfile` global ignores (git-present already covers + * them) and the "a re-include cannot resurrect a file under an excluded parent" + * rule (we favor inclusion instead). + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; + +/** Whether a compiled pattern set matched a path, and if so with which polarity. */ +export type GitignoreDecision = "ignore" | "include" | "unmatched"; + +/** One compiled gitignore rule (a single non-comment, non-blank line). */ +export interface GitignoreRule { + /** A `!`-prefixed re-include rule. */ + readonly negate: boolean; + /** A trailing-`/` directory-only rule (matches a directory and its contents). */ + readonly dirOnly: boolean; + /** The compiled matcher over a base-relative, forward-slashed path. */ + readonly regex: RegExp; +} + +/** Escape a single literal character for use inside a RegExp body. */ +function escapeRegexChar(c: string): string { + return /[.+^${}()|[\]\\]/.test(c) ? `\\${c}` : c; +} + +/** + * Collapse redundant consecutive `**` glob runs before compilation. + * + * `**\/**\/` (and longer runs) are semantically a single `**\/`, and `***`+ is + * treated as `**`. Emitting one RegExp fragment per raw run would produce a + * chain of overlapping unbounded quantifiers (`(?:.*\/)?(?:.*\/)?...`) whose + * match cost against a deeply nested path is exponential: a crafted `.gitignore` + * in an untrusted, non-git bound repo could freeze the single-threaded daemon + * (the git-absent fallback walks every path through this matcher). Collapsing + * the runs preserves gitignore semantics and bounds the compiled pattern so no + * such catastrophic backtracking is possible. The collapse regexes themselves + * scan the bounded pattern linearly (each unit is fixed-width, no ambiguity). + */ +function collapseGlobStars(pattern: string): string { + return pattern.replace(/(?:\*\*\/)+/g, "**/").replace(/\*{3,}/g, "**"); +} + +/** + * Compile a single gitignore glob body (already stripped of any `!`, leading + * `/`, and trailing `/`) into a RegExp body, honoring `**`, `*`, `?`. A `*` + * never crosses a path separator; `**` does. + */ +function globBodyToRegex(rawPattern: string): string { + const pattern = collapseGlobStars(rawPattern); + let re = ""; + for (let i = 0; i < pattern.length; i++) { + const c = pattern[i] ?? ""; + if (c === "*") { + if (pattern[i + 1] === "*") { + i++; // consume the second '*' + if (pattern[i + 1] === "/") { + i++; // consume the '/' + re += "(?:.*/)?"; + } else { + re += ".*"; + } + } else { + re += "[^/]*"; + } + } else if (c === "?") { + re += "[^/]"; + } else { + re += escapeRegexChar(c); + } + } + return re; +} + +/** + * Compile one gitignore line into a {@link GitignoreRule}, or null for a blank + * line or a comment. `patternText` is a single line as written in a `.gitignore`. + */ +export function compileRule(patternText: string): GitignoreRule | null { + let line = patternText.replace(/\r$/, ""); + // Trailing unescaped whitespace is not significant in gitignore. + line = line.replace(/(? boolean { + const rules: GitignoreRule[] = []; + for (const p of patterns) { + const rule = compileRule(p); + if (rule !== null) rules.push(rule); + } + return (relPath: string, isDir = false): boolean => decideRules(rules, normalizeRel(relPath), isDir) === "ignore"; +} + +/** The `.git/info/exclude` path relative to a repo root. */ +export const GIT_INFO_EXCLUDE = ".git/info/exclude"; + +/** The per-directory gitignore filename. */ +export const GITIGNORE_FILE = ".gitignore"; + +/** A disk-backed gitignore predicate over a repo-relative, forward-slashed path. */ +export type DiskGitignore = (relPath: string, isDir?: boolean) => boolean; + +export interface DiskGitignoreOptions { + /** Read a file by absolute path; return null when it does not exist. Injectable for tests. */ + readonly readFile?: (p: string) => string | null; +} + +function ancestorDirs(relPath: string): string[] { + const parts = relPath.split("/"); + const dirs: string[] = [""]; + for (let i = 0; i < parts.length - 1; i++) { + dirs.push(parts.slice(0, i + 1).join("/")); + } + return dirs; // "", "packages", "packages/a", ... +} + +/** + * Build a disk-backed gitignore matcher for `root` that honors the root + * `.gitignore`, nested `.gitignore` files, and `.git/info/exclude`. Rules are + * loaded lazily and cached per directory. Precedence (highest first): the + * deepest applicable `.gitignore`, then shallower ones, then the root + * `.gitignore`, then `.git/info/exclude`; the first source with a definitive + * decision wins (matching git's last-match-wins over the ordered concatenation). + */ +export function createDiskGitignore(root: string, opts: DiskGitignoreOptions = {}): DiskGitignore { + const readFile = opts.readFile ?? defaultReadFileOrNull; + /** dir (relative to root, "" for root) -> compiled rules, or null when no `.gitignore` there. */ + const dirRules = new Map(); + let excludeRules: readonly GitignoreRule[] | null | undefined; + + function rulesForDir(dir: string): readonly GitignoreRule[] | null { + const cached = dirRules.get(dir); + if (cached !== undefined) return cached; + const path = dir === "" ? join(root, GITIGNORE_FILE) : join(root, dir, GITIGNORE_FILE); + const text = safeRead(readFile, path); + const compiled = text === null ? null : parseGitignore(text); + dirRules.set(dir, compiled); + return compiled; + } + + function rulesForExclude(): readonly GitignoreRule[] | null { + if (excludeRules === undefined) { + const text = safeRead(readFile, join(root, GIT_INFO_EXCLUDE)); + excludeRules = text === null ? null : parseGitignore(text); + } + return excludeRules; + } + + return (relPath: string, isDir = false): boolean => { + const rel = normalizeRel(relPath); + if (rel === "") return false; + // Deepest applicable directory first (highest precedence). + const dirs = ancestorDirs(rel).reverse(); + for (const dir of dirs) { + const rules = rulesForDir(dir); + if (rules === null) continue; + const subPath = dir === "" ? rel : rel.slice(dir.length + 1); + const decision = decideRules(rules, subPath, isDir); + if (decision !== "unmatched") return decision === "ignore"; + } + const exclude = rulesForExclude(); + if (exclude !== null) { + const decision = decideRules(exclude, rel, isDir); + if (decision !== "unmatched") return decision === "ignore"; + } + return false; + }; +} + +function safeRead(readFile: (p: string) => string | null, path: string): string | null { + try { + return readFile(path); + } catch { + return null; + } +} + +function defaultReadFileOrNull(p: string): string | null { + // Any failure (missing/unreadable) reads as "no such gitignore", never a throw. + try { + return readFileSync(p, "utf8"); + } catch { + return null; + } +} diff --git a/src/registration/ignore.ts b/src/registration/ignore.ts index f8b3e42..63f1e28 100644 --- a/src/registration/ignore.ts +++ b/src/registration/ignore.ts @@ -35,6 +35,7 @@ import { readFileSync } from "node:fs"; import { join } from "node:path"; import { spawnSync } from "node:child_process"; +import { createDiskGitignore, type DiskGitignore } from "./gitignore.js"; /** A predicate over a repo-relative (forward-slashed) path: true means "drop, never register". */ export type IgnorePredicate = (relPath: string) => boolean; @@ -224,6 +225,15 @@ export interface SharedIgnoreOptions { * Defaults to a no-op. */ readonly onGitError?: (reason: string) => void; + /** + * PRD-019d: the dependency-free `.gitignore` matcher consulted ONLY when git + * is GENUINELY ABSENT (no `.git`, no `git` on PATH). Default: a disk-backed + * matcher over `root`'s `.gitignore` / nested `.gitignore` / `.git/info/exclude`. + * Injectable for tests. Not consulted when git is present (the snapshot is + * authoritative) nor when git is present-but-erroring (that degradation stays + * loud via {@link onGitError}; the parser must not silently mask it). + */ + readonly gitignoreFallback?: DiskGitignore; } /** @@ -239,6 +249,16 @@ export function createSharedIgnore(root: string, opts: SharedIgnoreOptions = {}) const gitLsFiles = opts.gitLsFiles ?? runGitLsFiles; const gitCheckIgnore = opts.gitCheckIgnore ?? runGitCheckIgnore; const onGitError = opts.onGitError ?? (() => {}); + // PRD-019d: the git-absent gitignore fallback, built lazily so a git-present + // workspace never reads `.gitignore` off disk (the snapshot already reflects it). + let gitignoreFallbackInstance: DiskGitignore | undefined; + function gitignoreFallback(relPath: string): boolean { + if (opts.gitignoreFallback !== undefined) return opts.gitignoreFallback(relPath); + if (gitignoreFallbackInstance === undefined) { + gitignoreFallbackInstance = createDiskGitignore(root, opts.readFile ? { readFile: opts.readFile } : {}); + } + return gitignoreFallbackInstance(relPath); + } /** Tracked+untracked-not-ignored snapshot; null when git is absent or no snapshot has ever succeeded. */ let eligible: Set | null = null; @@ -280,7 +300,15 @@ export function createSharedIgnore(root: string, opts: SharedIgnoreOptions = {}) refresh(); // NEC-007 point 1: warm the cache once at construction. function isGitIgnored(relPath: string): boolean { - if (eligible === null) return false; // no usable snapshot (git absent, or errored with nothing cached yet) + if (eligible === null) { + // PRD-019d: git GENUINELY ABSENT (no error recorded) - consult the + // dependency-free `.gitignore` parser so a bound non-git subtree is not + // gitignore-blind. Git present-but-ERRORING (lastError set, nothing + // cached) keeps the pre-019d behavior: the degradation stays loud via + // `onGitError` and is NOT silently masked by the parser (d-AC-5). + if (lastError === null) return gitignoreFallback(relPath); + return false; // errored with nothing cached yet + } if (eligible.has(relPath)) return false; // definitely NOT ignored // Cache miss against a real (possibly stale) snapshot: either genuinely // ignored, or created since the last refresh. The per-path fallback keeps diff --git a/src/registration/project-context.ts b/src/registration/project-context.ts new file mode 100644 index 0000000..3ec01f2 --- /dev/null +++ b/src/registration/project-context.ts @@ -0,0 +1,246 @@ +/** + * A per-project brood + watch running context (PRD-019a). + * + * Encapsulates ONE bound project's live machinery - the update-on-change + * registration pipeline (a {@link StoreBridge} + {@link RegistrationService}) and + * the auto-brood trigger - parameterized by `(root, tenancy, store)` instead of + * the daemon's former module-level single root. The multi-root supervisor + * (`src/project-supervisor.ts`) stands one of these up per active project and + * tears it down on unbind / brooding-off, so discovery for one project never + * enumerates paths under another (each context is rooted at its own bound path + * and scoped to its own tenancy project id). + * + * This mirrors the single-root wiring in `src/daemon.ts` (`buildRegistration` + + * `triggerAutoBrood`) rather than duplicating its logic inline in the daemon; the + * ignore predicate, the disk fs, and the brood runner are the same shared pieces. + * Each context owns its OWN brood guard so two projects can brood concurrently + * without one blocking the other. + */ +import { createBroodGuard } from "../brood-guard.js"; +import type { Timer } from "../poll-loop.js"; +import type { Tenancy } from "../hive-graph/model.js"; +import type { AsyncHiveGraphStore } from "../hive-graph/store.js"; +import type { PipelineMetricsSink } from "../telemetry/index.js"; +import { createDiskRegistrationFs } from "./disk-fs.js"; +import { createSharedIgnore, type IgnorePredicate, type SharedIgnore } from "./ignore.js"; +import { RegistrationService, type RegistrationFs } from "./service.js"; +import type { WatcherState } from "./fs-watch.js"; +import { StoreBridge } from "./store-bridge.js"; +import { createTlshFuzzyStep, DEFAULT_TUNABLE_FUZZY_CONFIG, type FuzzyConfig } from "./tlsh.js"; +import type { PendingReviewStore } from "./review-store.js"; +import { + discoverFiles, + evaluateAutoBroodAsync, + prepareFiles, + runBroodAsync, + shouldAutoBrood, + type AsyncBroodConfig, + type AsyncBroodRuntimeDeps, + type BroodResult, + type BroodRunOptions, +} from "../brooding/index.js"; +import { runBootProjectionLoad } from "../daemon.js"; +import { projectionFinalPath } from "../projection/write.js"; +import { DEFAULT_PROJECTION_REL_PATH } from "../projection/format.js"; +import type { InheritRow } from "../projection/inherit.js"; +import type { RunningContext } from "../project-supervisor.js"; +import type { ResolvedProject } from "../hive-graph/active-projects.js"; + +export interface ProjectContextDeps { + readonly project: ResolvedProject; + readonly tenancy: Tenancy; + /** The durable async store the watch pipeline persists to and auto-brood evaluates against. */ + readonly store: AsyncHiveGraphStore; + /** Auto-brood runtime deps (describe transport, embed provider). Absent -> auto-brood is skipped. */ + readonly broodDeps?: AsyncBroodRuntimeDeps; + /** Extra async brood config (never overrides store/tenancy/root/fs/isIgnored). */ + readonly broodConfig?: Partial>; + readonly broodOptions?: BroodRunOptions; + /** Disable auto-brood for this context (still watches). Default: enabled when broodDeps is present. */ + readonly autoBroodEnabled?: boolean; + /** + * PRD-011b AC-6, per project (PRD-019 remediation): on `start()`, load + + * validate this root's own `/.honeycomb/nectars.json` and inherit + * hash-matched files into the durable store with ZERO LLM calls, mirroring + * what the single-root daemon's `runBootProjectionLoad` did for the one cwd + * root. Fail-soft: a missing/invalid projection is skipped with no error. + * Default: enabled; set false to skip (tests that want no disk scan). + */ + readonly projectionPreWarm?: boolean; + /** Registration fs override (default: `createDiskRegistrationFs(root, isIgnored)`). */ + readonly registrationFs?: RegistrationFs; + /** Shared ignore override (default: `createSharedIgnore(root)`). */ + readonly sharedIgnore?: SharedIgnore; + readonly reviews?: PendingReviewStore; + readonly fuzzyConfig?: FuzzyConfig; + readonly timer?: Timer; + readonly debounceMs?: number; + readonly metrics?: PipelineMetricsSink; + /** Wrap the durable store (e.g. with telemetry) before auto-brood runs against it. Default: identity. */ + readonly wrapBroodStore?: (store: AsyncHiveGraphStore) => AsyncHiveGraphStore; + readonly log?: (line: Record) => void; + /** Observe watcher liveness changes (surfaced on `/health`). */ + readonly onWatcherStateChange?: (state: WatcherState) => void; + /** Observe a durable-flush failure from the bridge. */ + readonly onFlushError?: (err: unknown, op: string) => void; + /** The brood runner (default: {@link runBroodAsync}); injectable for tests. */ + readonly broodRun?: ( + config: AsyncBroodConfig, + deps?: AsyncBroodRuntimeDeps, + options?: BroodRunOptions, + ) => Promise; +} + +/** + * Build a {@link RunningContext} for one project. Construction is side-effect + * light (no watcher, no brood); `start()` runs auto-brood, then hydrates the + * bridge, starts the watcher, and requests the single cold-catch-up resync; + * `stop()` stops the watcher and drains the bridge writes before the context is + * released (a-AC-7). + */ +export function createProjectContext(deps: ProjectContextDeps): RunningContext { + const { project, tenancy, store } = deps; + const root = project.path; + const log = deps.log ?? (() => {}); + const sharedIgnore = deps.sharedIgnore ?? createSharedIgnore(root); + const isIgnored: IgnorePredicate = (relPath: string) => sharedIgnore.isIgnored(relPath); + const fs = deps.registrationFs ?? createDiskRegistrationFs(root, isIgnored); + const broodGuard = createBroodGuard(); + + let watcher: WatcherState = "stopped"; + let service: RegistrationService | null = null; + let bridge: StoreBridge | null = null; + let closed = false; + + /** + * The per-project projection pre-warm (PRD-011b AC-6 in the multi-root + * world): validate `/.honeycomb/nectars.json` against THIS project's + * tenancy and inherit hash-matched files into the durable store, so a fresh + * clone recalls without a brood. The disk scan runs only when the file + * validates (runBootProjectionLoad defers `diskHashes` behind validation). + */ + async function runProjectionPreWarm(): Promise { + if (deps.projectionPreWarm === false) return; + try { + await runBootProjectionLoad({ + tenancy, + filePath: projectionFinalPath(root, DEFAULT_PROJECTION_REL_PATH), + diskHashes: () => { + const discovery = discoverFiles({ root, fs, isIgnored }); + const prepared = prepareFiles(fs, discovery.files); + return new Map(prepared.map((p) => [p.file.relPath, p.contentHash] as const)); + }, + existingNectars: async () => { + const latest = await store.listLatestVersions(tenancy); + return new Set(latest.map((lv) => lv.identity.nectar)); + }, + write: async (rows: readonly InheritRow[]) => { + for (const row of rows) { + if ((await store.getIdentity(row.identity.nectar)) === undefined) { + await store.insertIdentity(row.identity); + } + await store.appendVersion(row.version); + } + }, + }); + } catch (err) { + // fail-soft: recall is simply not pre-warmed; the context still starts. + log({ level: "error", scope: "project-context.projection", projectId: project.projectId, err: String(err) }); + } + } + + async function runAutoBrood(): Promise { + const broodDeps = deps.broodDeps; + if (broodDeps === undefined) return; + if ((deps.autoBroodEnabled ?? true) === false) return; + if (!broodGuard.tryAcquire()) return; + try { + if (!shouldAutoBrood(await evaluateAutoBroodAsync(store, tenancy, root))) return; + const wrapped = deps.wrapBroodStore ? deps.wrapBroodStore(store) : store; + const config: AsyncBroodConfig = { + isIgnored, + ...deps.broodConfig, + store: wrapped, + tenancy, + root, + fs, + }; + const run = deps.broodRun ?? runBroodAsync; + await run(config, broodDeps, deps.broodOptions ?? {}); + } catch (err) { + log({ level: "error", scope: "project-context.brood", projectId: project.projectId, err: String(err) }); + } finally { + broodGuard.release(); + } + } + + return { + projectId: project.projectId, + path: root, + watcherState: () => watcher, + async start(): Promise { + if (closed) return; + // Projection pre-warm first (inherit hash-matched files with zero LLM + // calls, so a fresh clone's auto-brood evaluation sees the inherited + // rows), then brood (so a first boot's brood never races the watcher + // into a double mint), then hydrate + watch + cold-catch-up resync. + await runProjectionPreWarm(); + if (closed) return; + await runAutoBrood(); + if (closed) return; + bridge = new StoreBridge({ + durable: store, + onFlushError: (err, op) => { + deps.onFlushError?.(err, op); + log({ level: "error", scope: "project-context.bridge", projectId: project.projectId, op, err: String(err) }); + }, + }); + service = new RegistrationService({ + store: bridge, + tenancy, + fs, + root, + fuzzy: createTlshFuzzyStep(deps.fuzzyConfig ?? DEFAULT_TUNABLE_FUZZY_CONFIG), + isIgnored, + ...(deps.reviews !== undefined ? { pendingReviews: deps.reviews } : {}), + ...(deps.timer !== undefined ? { timer: deps.timer } : {}), + ...(deps.debounceMs !== undefined ? { debounceMs: deps.debounceMs } : {}), + ...(deps.metrics !== undefined ? { metrics: deps.metrics } : {}), + log, + onResyncRequested: () => sharedIgnore.refresh(), + onWatcherStateChange: (state: WatcherState) => { + watcher = state; + deps.onWatcherStateChange?.(state); + }, + }); + try { + await bridge.hydrate(tenancy); + } catch (err) { + log({ level: "error", scope: "project-context.hydrate", projectId: project.projectId, err: String(err) }); + } + if (closed) return; + service.start(); + watcher = "running"; + service.requestResync(); + }, + async stop(): Promise { + closed = true; + if (service !== null) { + service.stop(); + try { + await service._waitForIdle(); + } catch { + // best-effort drain + } + } + if (bridge !== null) { + try { + await bridge.whenFlushed(); + } catch { + // best-effort drain + } + } + watcher = "stopped"; + }, + }; +} diff --git a/src/service/argv.ts b/src/service/argv.ts index 1f593c0..edde39a 100644 --- a/src/service/argv.ts +++ b/src/service/argv.ts @@ -79,7 +79,11 @@ export function installCommands(plan: ServicePlan, uid: number): readonly Servic ]; } case "sc": { - const binPath = `"${process.execPath}" "${plan.execPath}" daemon`; + const binPath = + plan.apiaryHome === undefined + ? `"${process.execPath}" "${plan.execPath}" daemon` + : `cmd.exe /d /s /c "set ""APIARY_HOME=${plan.apiaryHome.replaceAll('"', '""')}"" && ` + + `""${process.execPath.replaceAll('"', '""')}"" ""${plan.execPath.replaceAll('"', '""')}"" daemon"`; return [ { command: "sc", args: ["create", WINDOWS_TASK_NAME, `binPath=${binPath}`, "start=", "auto"] }, { command: "sc", args: ["start", WINDOWS_TASK_NAME] }, diff --git a/src/service/index.ts b/src/service/index.ts index 360867c..e31f6a9 100644 --- a/src/service/index.ts +++ b/src/service/index.ts @@ -39,6 +39,10 @@ import { } from "./platform.js"; import { renderUnit, launchdLogDir } from "./templates.js"; +function serviceStateDir(plan: Pick): string { + return plan.apiaryHome !== undefined ? `${plan.apiaryHome}/nectar` : `${plan.home}/.apiary/nectar`; +} + /** A coarse, classified service status (what `nectar service-status` reports). */ export type ServiceStatus = "running" | "not-running" | "unknown"; @@ -217,15 +221,15 @@ export function createServiceModule(deps: ServiceModuleDeps): ServiceModule { if (needsFile) { try { if (p.manager === "schtasks" && unitTarget === "") { - unitTarget = `${p.home}/.honeycomb/nectar/nectar-task.xml`; + unitTarget = `${serviceStateDir(p)}/nectar-task.xml`; } fs.mkdirp(dirname(unitTarget)); fs.writeFile(unitTarget, renderUnit(p)); // NEC-042 item 2 / AC-018l.9: launchd writes stdout/stderr into - // `/.honeycomb/nectar`, but the plist lives under LaunchAgents, + // `/.apiary/nectar/logs`, but the plist lives under LaunchAgents, // so mkdirp(dirname(unitTarget)) above only created LaunchAgents. Create // the log directory too, or the daemon's macOS logs are silently lost. - if (p.manager === "launchd") fs.mkdirp(launchdLogDir(p.home)); + if (p.manager === "launchd") fs.mkdirp(launchdLogDir(p)); } catch (error) { return { ok: false, @@ -272,7 +276,7 @@ export function createServiceModule(deps: ServiceModuleDeps): ServiceModule { const { allOk, firstFailure, firstFailureResult } = await runAll(runner, uninstallCommands(p, uid)); - const stagedXml = p.manager === "schtasks" ? `${p.home}/.honeycomb/nectar/nectar-task.xml` : ""; + const stagedXml = p.manager === "schtasks" ? `${serviceStateDir(p)}/nectar-task.xml` : ""; try { if (p.unitPath !== "") fs.removeFile(p.unitPath); if (stagedXml !== "") fs.removeFile(stagedXml); diff --git a/src/service/platform.ts b/src/service/platform.ts index b489532..c15f2a5 100644 --- a/src/service/platform.ts +++ b/src/service/platform.ts @@ -71,6 +71,12 @@ export interface ServiceEnvironment { * we prefer a user unit unless the operator explicitly asked for a system one. */ readonly preferSystemScope?: boolean; + /** + * Optional installer-pinned fleet root. When set, templates/argv render + * `APIARY_HOME` into the service environment so runtime path resolution + * matches install-time intent. + */ + readonly apiaryHome?: string; } /** The fully-resolved plan: which manager, which scope, and the unit's on-disk location. */ @@ -91,6 +97,15 @@ export interface ServicePlan { readonly execPath: string; /** The home dir (units reference it for logs / working dir). */ readonly home: string; + /** Optional installer-pinned fleet root written into service env. */ + readonly apiaryHome?: string; +} + +function nonBlank(value: string | undefined): string | undefined { + if (value === undefined) return undefined; + const trimmed = value.trim(); + if (trimmed === "") return undefined; + return trimmed; } /** Gather the real {@link ServiceEnvironment} at the edge (the one impure call site). */ @@ -101,6 +116,7 @@ export function resolveServiceContext(execPath: string, preferSystemScope = fals privileged: isPrivileged(), execPath, preferSystemScope, + apiaryHome: nonBlank(process.env.APIARY_HOME), }; } @@ -214,5 +230,6 @@ export function resolveServicePlan(env: ServiceEnvironment): ServicePlan { label: SERVICE_LABEL, execPath: env.execPath, home: env.home, + ...(env.apiaryHome !== undefined ? { apiaryHome: env.apiaryHome } : {}), }; } diff --git a/src/service/templates.ts b/src/service/templates.ts index 67b1120..7cdf3c4 100644 --- a/src/service/templates.ts +++ b/src/service/templates.ts @@ -24,13 +24,14 @@ export const NECTAR_RUN_COMMAND = "daemon" as const; /** * The directory the launchd unit writes stdout/stderr logs into - * (`/.honeycomb/nectar`). Single-sourced here so the install path + * (`/nectar/logs`). Single-sourced here so the install path * (`service/index.ts`) creates exactly the directory the plist references, * rather than the daemon silently losing its logs on macOS (NEC-042 item 2 / * AC-018l.9). Forward slashes match the plist's own path style. */ -export function launchdLogDir(home: string): string { - return `${home}/.honeycomb/nectar`; +export function launchdLogDir(plan: Pick): string { + const stateDir = plan.apiaryHome !== undefined ? `${plan.apiaryHome}/nectar` : `${plan.home}/.apiary/nectar`; + return `${stateDir}/logs`; } /** @@ -85,8 +86,17 @@ export function escapeXml(value: string): string { export function renderLaunchdPlist(plan: ServicePlan): string { const node = escapeXml(process.execPath); const exec = escapeXml(plan.execPath); - const logDir = escapeXml(launchdLogDir(plan.home)); + const logDir = escapeXml(launchdLogDir(plan)); const label = escapeXml(plan.label); + const envBlock = + plan.apiaryHome === undefined + ? "" + : `\tEnvironmentVariables +\t +\t\tAPIARY_HOME +\t\t${escapeXml(plan.apiaryHome)} +\t +`; return ` @@ -99,6 +109,7 @@ export function renderLaunchdPlist(plan: ServicePlan): string { ${exec} ${NECTAR_RUN_COMMAND} +${envBlock} RunAtLoad KeepAlive @@ -130,6 +141,11 @@ export function renderSystemdUnit(plan: ServicePlan): string { // tokens; the run subcommand is a fixed literal with no spaces. const node = quoteSystemdToken(process.execPath); const exec = quoteSystemdToken(plan.execPath); + const envLine = + plan.apiaryHome === undefined + ? "" + : `Environment=${quoteSystemdToken(`APIARY_HOME=${plan.apiaryHome}`)} +`; return `[Unit] Description=nectar - semantic memory layer daemon Documentation=https://get.theapiary.sh @@ -142,6 +158,7 @@ Type=simple ExecStart=${node} ${exec} ${NECTAR_RUN_COMMAND} Restart=always RestartSec=${RESTART_SEC} +${envLine} [Install] WantedBy=default.target @@ -160,8 +177,18 @@ WantedBy=default.target * Scheduler rejects sub-minute intervals. */ export function renderScheduledTaskXml(plan: ServicePlan): string { - const node = escapeXml(process.execPath); - const exec = escapeXml(plan.execPath); + const node = process.execPath; + const exec = plan.execPath; + const windowsCommand = + plan.apiaryHome === undefined + ? { command: node, args: `"${exec}" ${NECTAR_RUN_COMMAND}` } + : { + command: "cmd.exe", + args: `/d /s /c "set ""APIARY_HOME=${plan.apiaryHome.replaceAll('"', '""')}"" && ` + + `""${node.replaceAll('"', '""')}"" ""${exec.replaceAll('"', '""')}"" ${NECTAR_RUN_COMMAND}"`, + }; + const command = escapeXml(windowsCommand.command); + const args = escapeXml(windowsCommand.args); return ` @@ -196,8 +223,8 @@ export function renderScheduledTaskXml(plan: ServicePlan): string { - ${node} - "${exec}" ${NECTAR_RUN_COMMAND} + ${command} + ${args} diff --git a/src/state-migration.ts b/src/state-migration.ts new file mode 100644 index 0000000..438e3b0 --- /dev/null +++ b/src/state-migration.ts @@ -0,0 +1,210 @@ +import { copyFileSync, existsSync, mkdirSync, renameSync, rmSync, writeFileSync } from "node:fs"; +import { basename, dirname, join } from "node:path"; +import { legacyRuntimeDir, nectarStateDir } from "./apiary-root.js"; +import type { RuntimeConfig } from "./config.js"; +import { defaultDoctorRegistryPath, registerWithDoctor } from "./doctor-registry.js"; +import { DaemonAlreadyRunningError } from "./errors.js"; +import { isPidAlive, readPidFile } from "./lock.js"; +import { TELEMETRY_DB_FILE_NAME, TELEMETRY_DIR_NAME } from "./telemetry/db.js"; + +export const MIGRATION_MARKER_FILE_NAME = ".migrated-from-honeycomb.json"; + +export const MIGRATION_RELATIVE_PATHS: readonly string[] = [ + "nectar.json", + "pending-reviews.json", + "telemetry/nectar.sqlite", + "nectar-usage-telemetry.json", +]; + +export interface ResolveStateReadPathOptions { + readonly runtimeDir?: string; + readonly legacyDir?: string; + readonly env?: NodeJS.ProcessEnv; +} + +/** + * Resolve a state file for reads during the migration window. + * New path wins; legacy path is used only when the new path is absent. + */ +export function resolveStateReadPath(relativePath: string, options: ResolveStateReadPathOptions = {}): string { + const runtimeDir = options.runtimeDir ?? nectarStateDir(options.env ?? process.env); + const legacyDir = options.legacyDir ?? legacyRuntimeDir(); + const preferred = join(runtimeDir, relativePath); + if (existsSync(preferred)) return preferred; + const legacy = join(legacyDir, relativePath); + if (existsSync(legacy)) return legacy; + return preferred; +} + +export interface LegacyInstanceGuardOptions { + readonly runtimeDir: string; + readonly pidFilePath: string; + readonly lockFilePath: string; + readonly legacyDir?: string; +} + +/** + * During the compatibility window, refuse boot when a legacy-path daemon PID is live. + */ +export function assertNoLegacyDaemonRunning(options: LegacyInstanceGuardOptions): void { + const legacyDir = options.legacyDir ?? legacyRuntimeDir(); + if (options.runtimeDir === legacyDir) return; + + const legacyPidPath = join(legacyDir, basename(options.pidFilePath)); + const legacyPid = readPidFile(legacyPidPath); + if (legacyPid === null || !isPidAlive(legacyPid)) return; + + const legacyLockPath = join(legacyDir, basename(options.lockFilePath)); + throw new DaemonAlreadyRunningError(legacyPid, legacyLockPath); +} + +export interface StateMigrationOptions { + readonly config: Pick; + readonly log?: (line: Record) => void; + readonly env?: NodeJS.ProcessEnv; + readonly nowIso?: () => string; + readonly legacyDir?: string; + readonly migrateFile?: (sourcePath: string, targetPath: string) => void; + readonly homeDir?: string; +} + +export interface StateMigrationResult { + readonly markerPath: string; + readonly moved: readonly string[]; + readonly failed: readonly string[]; + readonly refreshedRegistry: boolean; + readonly skipped: boolean; +} + +function anyLegacyStateExists(legacyDir: string): boolean { + for (const relativePath of MIGRATION_RELATIVE_PATHS) { + if (existsSync(join(legacyDir, relativePath))) return true; + } + return false; +} + +function copyThenRename(sourcePath: string, targetPath: string): void { + const targetDir = dirname(targetPath); + mkdirSync(targetDir, { recursive: true, mode: 0o700 }); + const tmpPath = `${targetPath}.${process.pid}.${Date.now()}.tmp`; + try { + copyFileSync(sourcePath, tmpPath); + renameSync(tmpPath, targetPath); + } catch (error) { + rmSync(tmpPath, { force: true }); + throw error; + } +} + +function writeMigrationMarker( + markerPath: string, + moved: readonly string[], + failed: readonly string[], + refreshedRegistry: boolean, + nowIso: () => string, +): void { + const body = { + schemaVersion: 1, + migratedAt: nowIso(), + moved, + failed, + refreshedRegistry, + }; + writeFileSync(markerPath, `${JSON.stringify(body, null, 2)}\n`, "utf8"); +} + +/** + * One-time additive migration from the legacy runtime dir into the new nectar state dir. + */ +export function runStateMigration(options: StateMigrationOptions): StateMigrationResult { + const runtimeDir = options.config.runtimeDir; + const legacyDir = options.legacyDir ?? legacyRuntimeDir(); + const markerPath = join(runtimeDir, MIGRATION_MARKER_FILE_NAME); + const markerExists = existsSync(markerPath); + const legacyExists = anyLegacyStateExists(legacyDir); + + mkdirSync(runtimeDir, { recursive: true, mode: 0o700 }); + + if (markerExists && !legacyExists) { + return { markerPath, moved: [], failed: [], refreshedRegistry: false, skipped: true }; + } + + const moved: string[] = []; + const failed: string[] = []; + for (const relativePath of MIGRATION_RELATIVE_PATHS) { + const sourcePath = join(legacyDir, relativePath); + const targetPath = join(runtimeDir, relativePath); + if (!existsSync(sourcePath) || existsSync(targetPath)) continue; + + try { + const migrateFile = options.migrateFile ?? copyThenRename; + migrateFile(sourcePath, targetPath); + rmSync(sourcePath, { force: true }); + moved.push(relativePath); + options.log?.({ + level: "info", + scope: "state-migration", + msg: "migrated file", + relativePath, + from: sourcePath, + to: targetPath, + }); + } catch (error) { + failed.push(relativePath); + options.log?.({ + level: "warn", + scope: "state-migration", + msg: "failed to migrate file", + relativePath, + from: sourcePath, + to: targetPath, + err: error instanceof Error ? error.message : String(error), + }); + } + } + + const shouldRefreshRegistry = legacyExists || moved.length > 0 || failed.length > 0; + let refreshedRegistry = false; + if (shouldRefreshRegistry) { + const registryPath = defaultDoctorRegistryPath(options.homeDir, options.env ?? process.env); + // Advertise only what actually exists post-migration: when the telemetry + // SQLite move FAILED and the legacy DB is still the live one, the entry + // must point doctor at the legacy path (a pointer at a not-yet-existing + // new path would report a missing DB for the whole retry window). Heals on + // the retry boot that completes the move, which refreshes again. + const telemetryDbPath = resolveStateReadPath(join(TELEMETRY_DIR_NAME, TELEMETRY_DB_FILE_NAME), { + runtimeDir, + legacyDir, + }); + try { + registerWithDoctor({ + config: { + host: options.config.host, + port: options.config.port, + pidFilePath: options.config.pidFilePath, + }, + registryPath, + overrides: { telemetryDbPath }, + }); + refreshedRegistry = true; + } catch (error) { + // Boot-path fail-soft: a PRESENT-but-malformed registry file (which + // `registerWithDoctor` refuses to clobber, and which any fleet product's + // installer may have written) must not brick nectar's boot. Log the + // reason loudly (path + why), skip the refresh, and keep booting. The + // fail-loud-on-write posture is preserved for the `nectar install` verb, + // which surfaces this same error to the operator and exits non-zero. + options.log?.({ + level: "warn", + scope: "state-migration", + msg: "doctor registry refresh skipped; the registry file could not be safely edited (not clobbered)", + registryPath, + err: error instanceof Error ? error.message : String(error), + }); + } + } + + const nowIso = options.nowIso ?? (() => new Date().toISOString()); + writeMigrationMarker(markerPath, moved, failed, refreshedRegistry, nowIso); + return { markerPath, moved, failed, refreshedRegistry, skipped: false }; +} diff --git a/src/telemetry-usage/emit.ts b/src/telemetry-usage/emit.ts index ecdc926..a3c895c 100644 --- a/src/telemetry-usage/emit.ts +++ b/src/telemetry-usage/emit.ts @@ -25,19 +25,19 @@ * stub that scripts/bake-posthog-key.mjs rewrites in dist/ at release time * (plain tsc build, so there is no esbuild define mechanism here). * - * distinct_id is anonymous: the honeycomb installer's ~/.honeycomb/install-id - * when present (so the funnel correlates across the product family), else a - * random UUID minted once and persisted in the ledger file under nectar's - * runtime dir (~/.honeycomb by default, resolveConfig's RUNTIME_DIR_NAME). + * distinct_id is anonymous: installer-minted `/install-id` when + * present (legacy fallback: `~/.honeycomb/install-id`), else a random UUID + * minted once and persisted in the ledger file under nectar's runtime dir. * Never an email, an account id, a hostname, or a path. */ import { randomUUID } from "node:crypto"; import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; -import { arch, homedir, platform } from "node:os"; +import { arch, platform } from "node:os"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; -import { RUNTIME_DIR_NAME } from "../config.js"; +import { legacyRuntimeDir, nectarStateDir, resolveApiaryRoot } from "../apiary-root.js"; +import { resolveStateReadPath } from "../state-migration.js"; import { POSTHOG_HOST, POSTHOG_KEY } from "./posthog-key.js"; /** The pinned PostHog capture path. The full ingest URL is `${host}${POSTHOG_CAPTURE_PATH}`. */ @@ -112,11 +112,11 @@ export interface UsageFetchRequestInit { /** The injectable fetch seam so tests record the POST instead of hitting PostHog. */ export type UsageFetch = (url: string, init: UsageFetchRequestInit) => Promise; -/** The injectable seams. Production defaults are the global fetch, process.env, and ~/.honeycomb. */ +/** The injectable seams. Production defaults are the global fetch, process.env, and `/nectar`. */ export interface UsageEmitDeps { readonly fetch?: UsageFetch; readonly env?: NodeJS.ProcessEnv; - /** Override the state dir (tests point this at a temp dir). Default: `~/.honeycomb`. */ + /** Override the state dir (tests point this at a temp dir). Default: `/nectar`. */ readonly dir?: string; /** Override the baked key (tests force the keyed branch without a bake). */ readonly posthogKey?: string; @@ -125,6 +125,10 @@ export interface UsageEmitDeps { readonly timeoutMs?: number; /** Override the reported version (default: read from package.json). */ readonly version?: string; + /** Test seam: override the resolved fleet root for install-id reads. */ + readonly apiaryRootDir?: string; + /** Test seam: override the legacy runtime dir for install-id reads. */ + readonly legacyRuntimeDir?: string; } /** The on-disk ledger: the fallback distinct id, the dedupe entries, and the last-seen version. */ @@ -162,9 +166,9 @@ export function captureUrl(host: string): string { return `${host.replace(/\/+$/, "")}${POSTHOG_CAPTURE_PATH}`; } -/** The default state dir: nectar's runtime dir convention (shared `~/.honeycomb`). */ -function defaultStateDir(): string { - return join(homedir(), RUNTIME_DIR_NAME); +/** The default state dir: nectar's runtime dir (`/nectar`). */ +function defaultStateDir(env: NodeJS.ProcessEnv = process.env): string { + return nectarStateDir(env); } /** @@ -184,10 +188,10 @@ export function readPackageVersion(): string { } } -/** Load the ledger, treating a missing or corrupt file as a fresh one (fail-soft). */ -function loadLedger(dir: string): UsageLedger { +/** Load one ledger file, treating a missing/corrupt payload as fresh (fail-soft). */ +function loadLedgerFile(path: string): UsageLedger { try { - const raw = readFileSync(join(dir, USAGE_LEDGER_FILE_NAME), "utf8"); + const raw = readFileSync(path, "utf8"); const parsed = JSON.parse(raw) as Partial; return { ...(typeof parsed.distinctId === "string" ? { distinctId: parsed.distinctId } : {}), @@ -199,22 +203,49 @@ function loadLedger(dir: string): UsageLedger { } } +/** Load the ledger from new-first path resolution during the migration window. */ +function loadLedger(dir: string, env: NodeJS.ProcessEnv, useLegacyFallback: boolean): UsageLedger { + const path = useLegacyFallback + ? resolveStateReadPath(USAGE_LEDGER_FILE_NAME, { runtimeDir: dir, env }) + : join(dir, USAGE_LEDGER_FILE_NAME); + return loadLedgerFile(path); +} + /** Persist the ledger, creating the state dir (0o700) when needed. Throws on IO failure; callers swallow. */ function saveLedger(dir: string, ledger: UsageLedger): void { mkdirSync(dir, { recursive: true, mode: 0o700 }); writeFileSync(join(dir, USAGE_LEDGER_FILE_NAME), `${JSON.stringify(ledger, null, 2)}\n`, "utf8"); } -/** Read the installer's anonymous install-id when present and non-empty, else undefined. */ -function readInstallId(dir: string): string | undefined { +/** Read one install-id candidate file when present and non-empty. */ +function readInstallIdFile(path: string): string | undefined { try { - const raw = readFileSync(join(dir, INSTALL_ID_FILE_NAME), "utf8").trim(); + const raw = readFileSync(path, "utf8").trim(); return raw.length > 0 ? raw : undefined; } catch { return undefined; } } +/** Read install-id with new-root-first ordering and a legacy fallback. */ +function readInstallId( + stateDir: string, + env: NodeJS.ProcessEnv, + useLegacyFallback: boolean, + apiaryRootDir: string | undefined, + legacyDir: string | undefined, +): string | undefined { + if (!useLegacyFallback) return readInstallIdFile(join(stateDir, INSTALL_ID_FILE_NAME)); + const resolvedApiaryRoot = apiaryRootDir ?? resolveApiaryRoot(env); + const resolvedLegacyDir = legacyDir ?? legacyRuntimeDir(); + return ( + readInstallIdFile(join(resolvedApiaryRoot, INSTALL_ID_FILE_NAME)) ?? + readInstallIdFile(join(resolvedLegacyDir, INSTALL_ID_FILE_NAME)) ?? + // Compatibility seam for tests that inject an explicit temporary state dir. + readInstallIdFile(join(stateDir, INSTALL_ID_FILE_NAME)) + ); +} + /** The dedupe ledger key: plain event name, except updated which dedupes per version. */ function dedupeKey(event: UsageEventName, version: string): string { return event === "nectar_updated" ? `${event}@${version}` : event; @@ -240,13 +271,21 @@ export async function emitUsageEvent(event: UsageEventName, deps: UsageEmitDeps if (isOptedOut(deps.env ?? process.env)) return { sent: false, skipped: "opted_out" }; try { - const dir = deps.dir ?? defaultStateDir(); + const env = deps.env ?? process.env; + const dir = deps.dir ?? defaultStateDir(env); const version = deps.version ?? readPackageVersion(); - const ledger = loadLedger(dir); + const useLegacyFallback = deps.dir === undefined; + const ledger = loadLedger(dir, env, useLegacyFallback); // distinct_id preference: the installer's install-id when present, else a // UUID minted once and persisted in the ledger so it stays stable. - let distinctId = readInstallId(dir); + let distinctId = readInstallId( + dir, + env, + useLegacyFallback, + deps.apiaryRootDir, + deps.legacyRuntimeDir, + ); if (distinctId === undefined) { if (ledger.distinctId === undefined) { ledger.distinctId = randomUUID(); @@ -333,9 +372,11 @@ export interface DaemonStartTelemetry { */ export async function recordDaemonStart(deps: UsageEmitDeps = {}): Promise { try { - const dir = deps.dir ?? defaultStateDir(); + const env = deps.env ?? process.env; + const dir = deps.dir ?? defaultStateDir(env); const version = deps.version ?? readPackageVersion(); - const lastSeen = loadLedger(dir).lastSeenVersion; + const useLegacyFallback = deps.dir === undefined; + const lastSeen = loadLedger(dir, env, useLegacyFallback).lastSeenVersion; const firstRun = await emitUsageEvent("nectar_first_run", deps); @@ -346,7 +387,7 @@ export async function recordDaemonStart(deps: UsageEmitDeps = {}): Promise/nectar/telemetry/nectar.sqlite`. */ +export function defaultTelemetryDbPath(home: string = homedir(), env: NodeJS.ProcessEnv = process.env): string { + return telemetryDbPathForRuntimeDir(nectarStateDir(env, { home })); } /** diff --git a/test/active-projects.test.ts b/test/active-projects.test.ts new file mode 100644 index 0000000..81f063e --- /dev/null +++ b/test/active-projects.test.ts @@ -0,0 +1,151 @@ +/** + * PRD-019a: active-project resolution + the pathological-root guard + the + * `/health` slice builder. Pure functions - hermetic, no disk, no env mutation. + * Runs against the compiled `dist/` output. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { + activeProjectsHealth, + isPathologicalRoot, + resolveActiveProjects, +} from "../dist/hive-graph/active-projects.js"; +import { defaultBroodingState, withGlobalBrooding, withProjectBrooding } from "../dist/registration/brooding-state.js"; + +const HOME = "/home/op"; + +function bindings(...pairs: Array<[string, string]>): Array<{ path: string; projectId: string }> { + return pairs.map(([path, projectId]) => ({ path, projectId })); +} + +// ── index AC-1 / a-AC-1: dormant by default ───────────────────────────────── + +test("index AC-1 / a-AC-1 zero bindings resolves to zero active projects (dormant)", () => { + const resolution = resolveActiveProjects({ bindings: [], broodingState: defaultBroodingState(), home: HOME, platform: "linux" }); + assert.equal(resolution.active.length, 0); + assert.equal(resolution.projects.length, 0); + const health = activeProjectsHealth(resolution, () => "stopped"); + assert.equal(health.count, 0); + assert.equal(health.reason, "no-active-projects", "the machine-readable dormancy reason"); +}); + +// ── index AC-2 / a-AC-3: one bound, brooding-on project activates ──────────── + +test("index AC-2 / a-AC-3 one bound brooding-on project resolves to exactly one active project scoped to its projectId", () => { + const resolution = resolveActiveProjects({ + bindings: bindings(["/work/repo", "proj-1"]), + broodingState: defaultBroodingState(), + home: HOME, + platform: "linux", + }); + assert.equal(resolution.active.length, 1); + assert.equal(resolution.active[0]?.projectId, "proj-1"); + assert.equal(resolution.active[0]?.path, "/work/repo"); + assert.equal(resolution.active[0]?.brooding, "active"); + const health = activeProjectsHealth(resolution, () => "running"); + assert.equal(health.count, 1); + assert.equal(health.reason, null); + assert.equal(health.projects[0]?.watcher, "running"); +}); + +// ── index AC-3 / a-AC-4: two bound projects, independent ───────────────────── + +test("index AC-3 / a-AC-4 two bound brooding-on projects each resolve independently under their own projectId", () => { + const resolution = resolveActiveProjects({ + bindings: bindings(["/work/a", "proj-a"], ["/work/b", "proj-b"]), + broodingState: defaultBroodingState(), + home: HOME, + platform: "linux", + }); + assert.deepEqual( + resolution.active.map((p) => [p.projectId, p.path]).sort(), + [["proj-a", "/work/a"], ["proj-b", "/work/b"]].sort(), + ); +}); + +// ── b-AC / brooding state gates the active set ─────────────────────────────── + +test("a per-project OFF removes it from the active set but keeps it visible on /health as paused", () => { + const state = withProjectBrooding(defaultBroodingState(), "proj-off", "off"); + const resolution = resolveActiveProjects({ + bindings: bindings(["/work/on", "proj-on"], ["/work/off", "proj-off"]), + broodingState: state, + home: HOME, + platform: "linux", + }); + assert.deepEqual(resolution.active.map((p) => p.projectId), ["proj-on"]); + const off = resolution.projects.find((p) => p.projectId === "proj-off"); + assert.equal(off?.brooding, "paused"); +}); + +test("a global pause makes nothing active; /health reason is global-paused; per-project state reads global-paused", () => { + const state = withGlobalBrooding(defaultBroodingState(), "paused"); + const resolution = resolveActiveProjects({ + bindings: bindings(["/work/a", "proj-a"]), + broodingState: state, + home: HOME, + platform: "linux", + }); + assert.equal(resolution.active.length, 0); + assert.equal(resolution.globalPaused, true); + const health = activeProjectsHealth(resolution, () => "stopped"); + assert.equal(health.reason, "global-paused"); + assert.equal(health.projects[0]?.brooding, "global-paused"); +}); + +// ── a-AC-6 / pathological-root guard ───────────────────────────────────────── + +test("a-AC-6 a binding that resolves to $HOME / a filesystem root / System32 is refused, not activated", () => { + const resolution = resolveActiveProjects({ + bindings: bindings(["/home/op", "p-home"], ["/", "p-root"], ["C:/Windows/System32", "p-sys"], ["/work/ok", "p-ok"]), + broodingState: defaultBroodingState(), + home: HOME, + platform: "win32", + env: { WINDIR: "C:\\Windows" }, + }); + assert.deepEqual(resolution.active.map((p) => p.projectId), ["p-ok"], "only the safe binding activates"); + const refusedIds = resolution.refused.map((r) => r.projectId).sort(); + assert.deepEqual(refusedIds, ["p-home", "p-root", "p-sys"].sort()); + assert.ok(resolution.refused.every((r) => r.reason === "pathological-root")); + const health = activeProjectsHealth(resolution, () => "running"); + assert.equal(health.refused.length, 3); +}); + +test("isPathologicalRoot: $HOME, POSIX root, Windows drive root, and System32 are guarded; a normal path is not", () => { + // The guard is a pure function of (path, platform): win32-shaped fixtures + // must produce the same verdict regardless of the HOST the suite runs on. + assert.equal(isPathologicalRoot("/home/op", { home: "/home/op", platform: "linux" }), true); + assert.equal(isPathologicalRoot("/", { platform: "linux" }), true); + assert.equal(isPathologicalRoot("C:\\", { platform: "win32", home: "C:\\Users\\op" }), true); + assert.equal(isPathologicalRoot("C:\\Windows\\System32", { platform: "win32", env: { WINDIR: "C:\\Windows" }, home: "C:\\Users\\op" }), true); + assert.equal(isPathologicalRoot("/work/my-repo", { home: "/home/op", platform: "linux" }), false); + assert.equal(isPathologicalRoot("/home/op/projects/x", { home: "/home/op", platform: "linux" }), false); +}); + +test("isPathologicalRoot per-platform semantics: System32 is Windows-only, POSIX $HOME stays case-sensitive, win32 folds case", () => { + // The System32 leg must NOT false-positive on POSIX: a directory that merely + // spells /Windows/System32 is a legitimate bindable folder there. + assert.equal(isPathologicalRoot("/Windows/System32", { platform: "linux", home: "/home/op", env: { WINDIR: "C:\\Windows" } }), false); + assert.equal(isPathologicalRoot("C:\\Windows\\System32", { platform: "win32", home: "C:\\Users\\op" }), true, "win32 falls back to C:/Windows when WINDIR is unset"); + + // POSIX is case-sensitive: /home/OP is NOT the same directory as /home/op. + assert.equal(isPathologicalRoot("/home/OP", { home: "/home/op", platform: "linux" }), false); + // win32 folds case: c:\ and a differently-cased System32 are still guarded. + assert.equal(isPathologicalRoot("c:\\", { platform: "win32", home: "C:\\Users\\op" }), true); + assert.equal(isPathologicalRoot("c:\\WINDOWS\\system32", { platform: "win32", env: { WINDIR: "C:\\Windows" }, home: "C:\\Users\\op" }), true); + + // A trailing separator on the bound path does not evade the $HOME guard. + assert.equal(isPathologicalRoot("/home/op/", { home: "/home/op", platform: "linux" }), true); + assert.equal(isPathologicalRoot("C:\\Users\\op\\", { platform: "win32", home: "C:\\Users\\op" }), true); +}); + +test("de-duplicates bindings by projectId (first binding wins) and skips blank ids/paths", () => { + const resolution = resolveActiveProjects({ + bindings: bindings(["/work/a", "dup"], ["/work/a2", "dup"], ["", "blank-path"], ["/work/c", " "]), + broodingState: defaultBroodingState(), + home: HOME, + platform: "linux", + }); + assert.deepEqual(resolution.projects.map((p) => p.projectId), ["dup"]); + assert.equal(resolution.projects[0]?.path, "/work/a", "the first binding for a projectId wins"); +}); diff --git a/test/apiary-root.test.ts b/test/apiary-root.test.ts new file mode 100644 index 0000000..dd1d45a --- /dev/null +++ b/test/apiary-root.test.ts @@ -0,0 +1,64 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { join } from "node:path"; +import { nectarStateDir, resolveApiaryRoot } from "../dist/apiary-root.js"; +import { resolveConfig } from "../dist/config.js"; + +function withEnv(name: string, value: string | undefined, fn: () => void): void { + const prior = process.env[name]; + if (value === undefined) delete process.env[name]; + else process.env[name] = value; + try { + fn(); + } finally { + if (prior === undefined) delete process.env[name]; + else process.env[name] = prior; + } +} + +test("a-AC-1 resolveApiaryRoot defaults to /.apiary and nectarStateDir to /.apiary/nectar", () => { + const env: NodeJS.ProcessEnv = {}; + assert.equal(resolveApiaryRoot(env, { platform: "darwin", home: "/home/op" }), join("/home/op", ".apiary")); + assert.equal(nectarStateDir(env, { platform: "darwin", home: "/home/op" }), join("/home/op", ".apiary", "nectar")); +}); + +test("a-AC-2 APIARY_HOME wins when set and non-blank; blank is treated as unset", () => { + const envSet: NodeJS.ProcessEnv = { APIARY_HOME: " /custom/root " }; + assert.equal(resolveApiaryRoot(envSet, { platform: "darwin", home: "/home/op" }), "/custom/root"); + + const envBlank: NodeJS.ProcessEnv = { APIARY_HOME: " " }; + assert.equal(resolveApiaryRoot(envBlank, { platform: "darwin", home: "/home/op" }), join("/home/op", ".apiary")); +}); + +test("a-AC-3 Linux uses XDG_STATE_HOME only when explicitly set and non-blank", () => { + const linuxXdg: NodeJS.ProcessEnv = { XDG_STATE_HOME: "/xdg/state" }; + assert.equal(resolveApiaryRoot(linuxXdg, { platform: "linux", home: "/home/op" }), join("/xdg/state", "apiary")); + assert.equal(resolveApiaryRoot({}, { platform: "linux", home: "/home/op" }), join("/home/op", ".apiary")); + assert.equal( + resolveApiaryRoot({ XDG_STATE_HOME: "/xdg/state" }, { platform: "win32", home: "/home/op" }), + join("/home/op", ".apiary"), + ); +}); + +test("security: a relative APIARY_HOME or XDG_STATE_HOME is ignored (env roots honored only when absolute; never cwd-anchored)", () => { + const relApiary: NodeJS.ProcessEnv = { APIARY_HOME: "relative/root" }; + assert.equal(resolveApiaryRoot(relApiary, { platform: "linux", home: "/home/op" }), join("/home/op", ".apiary")); + + const relXdg: NodeJS.ProcessEnv = { XDG_STATE_HOME: "relative/state" }; + assert.equal(resolveApiaryRoot(relXdg, { platform: "linux", home: "/home/op" }), join("/home/op", ".apiary")); + + // Windows-shaped absolutes are still honored on any host (win32.isAbsolute superset). + const winAbs: NodeJS.ProcessEnv = { APIARY_HOME: "C:\\fleet\\root" }; + assert.equal(resolveApiaryRoot(winAbs, { platform: "win32", home: "C:\\Users\\op" }), "C:\\fleet\\root"); +}); + +test("a-AC-4 NECTAR_RUNTIME_DIR keeps precedence over APIARY_HOME for runtimeDir/pid/lock", () => { + withEnv("APIARY_HOME", "/custom/root", () => { + withEnv("NECTAR_RUNTIME_DIR", "/tmp/nectar-runtime", () => { + const cfg = resolveConfig(); + assert.equal(cfg.runtimeDir, "/tmp/nectar-runtime"); + assert.equal(cfg.pidFilePath, join("/tmp/nectar-runtime", "nectar.pid")); + assert.equal(cfg.lockFilePath, join("/tmp/nectar-runtime", "nectar.lock")); + }); + }); +}); diff --git a/test/brooding-state.test.ts b/test/brooding-state.test.ts new file mode 100644 index 0000000..f945c5e --- /dev/null +++ b/test/brooding-state.test.ts @@ -0,0 +1,113 @@ +/** + * PRD-019b: the nectar-owned brooding-state store. Hermetic - every test points + * the store at a temp dir under `os.tmpdir()` via the `dir` override, NEVER the + * real user home. Runs against the compiled `dist/` output. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + BROODING_STATE_FILE_NAME, + defaultBroodingState, + effectiveBrooding, + loadBroodingState, + writeBroodingState, + withGlobalBrooding, + withProjectBrooding, +} from "../dist/registration/brooding-state.js"; + +function tempDir(): string { + return mkdtempSync(join(tmpdir(), "nectar-brooding-")); +} + +test("b-AC-1 a missing file (and missing dir) reads as defaults: global on, a new project defaults to on; the first write creates the dir", () => { + const parent = tempDir(); + try { + const dir = join(parent, "does-not-exist-yet"); // not created + const state = loadBroodingState({ dir }); + assert.equal(state.globalBrooding, "on", "global defaults to on"); + assert.equal(effectiveBrooding(state, "proj-x"), "active", "a newly-seen project defaults to on -> active"); + assert.equal(existsSync(dir), false, "loading never creates the dir"); + + // First write creates the dir + file. + writeBroodingState(withProjectBrooding(state, "proj-x", "on"), { dir }); + assert.equal(existsSync(join(dir, BROODING_STATE_FILE_NAME)), true, "the first write creates the file (and its dir)"); + } finally { + rmSync(parent, { recursive: true, force: true }); + } +}); + +test("b-AC-2 a malformed / non-object file warns and falls back to defaults (never throws)", () => { + const dir = tempDir(); + try { + const warnings: string[] = []; + writeFileSync(join(dir, BROODING_STATE_FILE_NAME), "{ not json", "utf8"); + const malformed = loadBroodingState({ dir, warn: (m) => warnings.push(m) }); + assert.deepEqual(malformed, defaultBroodingState(), "malformed JSON -> defaults"); + assert.ok(warnings.length >= 1, "a warning was emitted"); + + writeFileSync(join(dir, BROODING_STATE_FILE_NAME), JSON.stringify([1, 2, 3]), "utf8"); + const nonObject = loadBroodingState({ dir, warn: () => {} }); + assert.deepEqual(nonObject, defaultBroodingState(), "a non-object payload -> defaults"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("unknown top-level keys and malformed per-project values warn and are skipped (forward compatibility)", () => { + const dir = tempDir(); + try { + const warnings: string[] = []; + writeFileSync( + join(dir, BROODING_STATE_FILE_NAME), + JSON.stringify({ + schemaVersion: 1, + globalBrooding: "paused", + futureKey: { anything: true }, + projects: { good: { brooding: "off" }, bad: { brooding: "maybe" }, junk: 7 }, + }), + "utf8", + ); + const state = loadBroodingState({ dir, warn: (m) => warnings.push(m) }); + assert.equal(state.globalBrooding, "paused"); + assert.equal(state.projects["good"], "off", "a valid project survives"); + assert.equal(state.projects["bad"], undefined, "an invalid brooding value is skipped"); + assert.equal(state.projects["junk"], undefined, "a malformed project entry is skipped"); + assert.ok(warnings.some((w) => w.includes("futureKey")), "the unknown key is warned"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("effectiveBrooding: global pause beats a per-project on; a per-project off pauses; else active", () => { + const base = defaultBroodingState(); + assert.equal(effectiveBrooding(base, "p"), "active"); + assert.equal(effectiveBrooding(withProjectBrooding(base, "p", "off"), "p"), "paused"); + const paused = withGlobalBrooding(withProjectBrooding(base, "p", "on"), "paused"); + assert.equal(effectiveBrooding(paused, "p"), "global-paused", "global pause wins over per-project on"); +}); + +test("b-AC-6 an atomic write round-trips; a write to an unwritable location throws so the caller can preserve prior state", () => { + const dir = tempDir(); + try { + const next = withProjectBrooding(withGlobalBrooding(defaultBroodingState(), "on"), "p1", "off"); + writeBroodingState(next, { dir }); + const reloaded = loadBroodingState({ dir }); + assert.equal(reloaded.projects["p1"], "off", "the write round-trips"); + assert.equal(reloaded.globalBrooding, "on"); + // The on-disk shape carries the per-project { brooding } object form. + const raw = JSON.parse(readFileSync(join(dir, BROODING_STATE_FILE_NAME), "utf8")); + assert.deepEqual(raw.projects.p1, { brooding: "off" }); + assert.equal(raw.schemaVersion, 1); + + // A write whose parent path is a FILE (not a dir) fails hard (mkdir throws), + // so the toggle API surfaces a 500 and preserves the prior state. + const filePath = join(dir, "as-a-file"); + writeFileSync(filePath, "x", "utf8"); + assert.throws(() => writeBroodingState(next, { dir: join(filePath, "nested") })); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/test/config.test.ts b/test/config.test.ts index a6ff330..1da9cdf 100644 --- a/test/config.test.ts +++ b/test/config.test.ts @@ -6,6 +6,7 @@ */ import { test } from "node:test"; import assert from "node:assert/strict"; +import { join } from "node:path"; import { resolveConfig, DEFAULT_PORT, DEFAULT_POLL_INTERVAL_MS } from "../dist/config.js"; import { ConfigError } from "../dist/errors.js"; @@ -76,3 +77,14 @@ test("EX-2 an explicit override bypasses env parsing (an ephemeral port 0 overri assert.equal(resolveConfig({ port: 0 }).port, 0); }); }); + +test("a-AC-2 resolveConfig default runtimeDir follows APIARY_HOME/nectar when APIARY_HOME is set", () => { + withEnv("APIARY_HOME", "/custom/root", () => { + withEnv("NECTAR_RUNTIME_DIR", undefined, () => { + const cfg = resolveConfig(); + assert.equal(cfg.runtimeDir, join("/custom/root", "nectar")); + assert.equal(cfg.pidFilePath, join("/custom/root", "nectar", "nectar.pid")); + assert.equal(cfg.lockFilePath, join("/custom/root", "nectar", "nectar.lock")); + }); + }); +}); diff --git a/test/daemon-active-projects.test.ts b/test/daemon-active-projects.test.ts new file mode 100644 index 0000000..e35fa1c --- /dev/null +++ b/test/daemon-active-projects.test.ts @@ -0,0 +1,169 @@ +/** + * PRD-019a: the daemon's multi-root, dormant-by-default active-project wiring, + * exercised end to end against a bound socket with a FAKE context factory and an + * injected (manually-driven) reconcile timer. Hermetic: an ephemeral port + a + * temp runtime dir; no disk under the real home, no network. Runs against the + * compiled `dist/` output. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { get } from "node:http"; +import { assembleDaemon } from "../dist/index.js"; +import type { ActiveProjectResolution, ResolvedProject } from "../dist/hive-graph/active-projects.js"; + +const silent = () => {}; + +/** A timer that captures scheduled callbacks without firing them (so reconcile is driven explicitly). */ +function inertTimer() { + return { set: (_fn: () => void, _ms: number) => 1, clear: (_h: unknown) => {} }; +} + +function fetchHealth(port: number): Promise<{ status: number; body: any }> { + return new Promise((resolve, reject) => { + get({ host: "127.0.0.1", port, path: "/health" }, (res) => { + const chunks: Buffer[] = []; + res.on("data", (c: Buffer) => chunks.push(c)); + res.on("end", () => resolve({ status: res.statusCode ?? 0, body: JSON.parse(Buffer.concat(chunks).toString("utf8")) })); + }).on("error", reject); + }); +} + +function tempRuntimeDir(): string { + return mkdtempSync(join(tmpdir(), "nectar-ap-")); +} + +test("index AC-1 / a-AC-1 / a-AC-2 a daemon with zero active projects broods nothing and /health reports activeProjects:0 reason no-active-projects", async () => { + const runtimeDir = tempRuntimeDir(); + let factoryCalls = 0; + const daemon = assembleDaemon({ + port: 0, + runtimeDir, + log: silent, + activeProjects: { + resolve: (): ActiveProjectResolution => ({ projects: [], refused: [], active: [], globalPaused: false }), + factory: () => { + factoryCalls += 1; + throw new Error("factory must not be called when there are no active projects"); + }, + timer: inertTimer(), + }, + }); + try { + const port = await daemon.start(); + const health = await fetchHealth(port); + assert.equal(health.body.activeProjects.count, 0); + assert.equal(health.body.activeProjects.reason, "no-active-projects"); + assert.equal(factoryCalls, 0, "no context is constructed => nothing is brooded/watched (never the cwd)"); + } finally { + await daemon.shutdown(); + rmSync(runtimeDir, { recursive: true, force: true }); + } +}); + +test("a-AC-5 / index AC-7 reconcileActiveProjects starts a newly-active project and stops it on unbind, updating /health", async () => { + const runtimeDir = tempRuntimeDir(); + const events: string[] = []; + let active: ResolvedProject[] = []; + const daemon = assembleDaemon({ + port: 0, + runtimeDir, + log: silent, + activeProjects: { + resolve: (): ActiveProjectResolution => ({ projects: active, refused: [], active, globalPaused: false }), + factory: (p: ResolvedProject) => { + let state: "stopped" | "running" = "stopped"; + return { + projectId: p.projectId, + path: p.path, + watcherState: () => (state === "running" ? "running" : "stopped"), + start: async () => { + events.push(`start:${p.projectId}`); + state = "running"; + }, + stop: async () => { + events.push(`stop:${p.projectId}`); + state = "stopped"; + }, + }; + }, + timer: inertTimer(), + }, + }); + try { + const port = await daemon.start(); + // Dormant at boot. + assert.equal((await fetchHealth(port)).body.activeProjects.count, 0); + + // Bind a project, then reconcile (as the 019b toggle API would). + active = [{ projectId: "proj-1", path: "/work/repo", brooding: "active" }]; + await daemon.reconcileActiveProjects(); + assert.deepEqual(events, ["start:proj-1"]); + let health = await fetchHealth(port); + assert.equal(health.body.activeProjects.count, 1); + assert.equal(health.body.activeProjects.projects[0].projectId, "proj-1"); + assert.equal(health.body.activeProjects.projects[0].watcher, "running"); + + // Unbind it; the next reconcile stops the context with no daemon restart. + active = []; + await daemon.reconcileActiveProjects(); + assert.deepEqual(events, ["start:proj-1", "stop:proj-1"]); + health = await fetchHealth(port); + assert.equal(health.body.activeProjects.count, 0); + assert.equal(health.body.activeProjects.reason, "no-active-projects"); + } finally { + await daemon.shutdown(); + rmSync(runtimeDir, { recursive: true, force: true }); + } +}); + +test("a-AC-6 a pathological bound root is surfaced on /health as refused: pathological-root and not started", async () => { + const runtimeDir = tempRuntimeDir(); + let factoryCalls = 0; + const resolution: ActiveProjectResolution = { + projects: [], + refused: [{ projectId: "p-home", path: "/home/op", reason: "pathological-root" }], + active: [], + globalPaused: false, + }; + const daemon = assembleDaemon({ + port: 0, + runtimeDir, + log: silent, + activeProjects: { + resolve: () => resolution, + factory: () => { + factoryCalls += 1; + throw new Error("a pathological root must never be activated"); + }, + timer: inertTimer(), + }, + }); + try { + const port = await daemon.start(); + const health = await fetchHealth(port); + assert.equal(health.body.activeProjects.count, 0); + assert.equal(health.body.activeProjects.refused.length, 1); + assert.equal(health.body.activeProjects.refused[0].reason, "pathological-root"); + assert.equal(factoryCalls, 0); + } finally { + await daemon.shutdown(); + rmSync(runtimeDir, { recursive: true, force: true }); + } +}); + +test("a legacy single-root daemon (no activeProjects option) is unaffected: /health still carries the default activeProjects slice", async () => { + const runtimeDir = tempRuntimeDir(); + const daemon = assembleDaemon({ port: 0, runtimeDir, log: silent }); + try { + const port = await daemon.start(); + const health = await fetchHealth(port); + assert.equal(health.body.activeProjects.count, 0); + assert.equal(health.body.activeProjects.reason, "no-active-projects"); + } finally { + await daemon.shutdown(); + rmSync(runtimeDir, { recursive: true, force: true }); + } +}); diff --git a/test/doctor-registry.test.ts b/test/doctor-registry.test.ts index 19ba48c..4fe64d6 100644 --- a/test/doctor-registry.test.ts +++ b/test/doctor-registry.test.ts @@ -1,6 +1,6 @@ import { test } from "node:test"; import assert from "node:assert/strict"; -import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; @@ -11,6 +11,7 @@ import { DEFAULT_RESTART_GIVE_UP_THRESHOLD, DEFAULT_RESTART_COOLDOWN_MS, DoctorRegistryError, + defaultDoctorRegistryPath, buildNectarRegistryEntry, registerWithDoctor, } from "../dist/doctor-registry.js"; @@ -20,13 +21,38 @@ function tmpDir() { return mkdtempSync(join(tmpdir(), "nectar-registry-")); } -const config = { host: "127.0.0.1", port: 3854, pidFilePath: "/home/op/.honeycomb/nectar.pid" }; +test("c-AC-3 defaultDoctorRegistryPath writes to /registry.json when the fleet root exists", () => { + const dir = tmpDir(); + try { + const fleetRoot = join(dir, ".apiary-root"); + const legacyHome = join(dir, "home"); + mkdirSync(fleetRoot, { recursive: true }); + const path = defaultDoctorRegistryPath(legacyHome, { APIARY_HOME: fleetRoot }); + assert.equal(path, join(fleetRoot, "registry.json")); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("c-AC-3 defaultDoctorRegistryPath falls back to legacy registry when the fleet root is absent", () => { + const dir = tmpDir(); + try { + const fleetRoot = join(dir, ".apiary-root-missing"); + const home = join(dir, "home"); + const path = defaultDoctorRegistryPath(home, { APIARY_HOME: fleetRoot }); + assert.equal(path, join(home, ".honeycomb", "doctor.daemons.json")); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +const config = { host: "127.0.0.1", port: 3854, pidFilePath: "/home/op/.apiary/nectar/nectar.pid" }; test("buildNectarRegistryEntry resolves healthUrl/pidPath from config and doctor's defaults", () => { const entry = buildNectarRegistryEntry(config); assert.equal(entry.name, NECTAR_DAEMON_NAME); assert.equal(entry.healthUrl, "http://127.0.0.1:3854/health"); - assert.equal(entry.pidPath, "/home/op/.honeycomb/nectar.pid"); + assert.equal(entry.pidPath, "/home/op/.apiary/nectar/nectar.pid"); assert.equal(entry.probeIntervalMs, DEFAULT_PROBE_INTERVAL_MS); assert.equal(entry.startupGraceMs, DEFAULT_STARTUP_GRACE_MS); assert.equal(entry.restartGiveUpThreshold, DEFAULT_RESTART_GIVE_UP_THRESHOLD); @@ -54,7 +80,7 @@ test("AC-018a.9 the registry entry nectar install writes marks the OS unit as re test("buildNectarRegistryEntry declares the absolute telemetry SQLite DB path, colocated with pidPath's runtime dir (AC-1 / AC-017a.1.1)", () => { const entry = buildNectarRegistryEntry(config); - assert.equal(entry.telemetryDbPath, join("/home/op/.honeycomb", TELEMETRY_DIR_NAME, TELEMETRY_DB_FILE_NAME)); + assert.equal(entry.telemetryDbPath, join("/home/op/.apiary/nectar", TELEMETRY_DIR_NAME, TELEMETRY_DB_FILE_NAME)); }); test("telemetryDbPath is overridable, mirroring the other per-daemon override fields", () => { diff --git a/test/fixes.test.ts b/test/fixes.test.ts index f54d8a5..a066fa7 100644 --- a/test/fixes.test.ts +++ b/test/fixes.test.ts @@ -7,7 +7,7 @@ */ import { test } from "node:test"; import assert from "node:assert/strict"; -import { chmodSync, mkdtempSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, statSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { get } from "node:http"; @@ -151,7 +151,7 @@ test("concurrent start() calls share one startup and return the same bound port" } }); -// ── PRD-018k / NEC-041: the ~/.honeycomb/nectar.json config-file loader ───────── +// ── PRD-018k / NEC-041: the /nectar/nectar.json config-file loader ── function writeNectarJson(dir: string, body: string): void { writeFileSync(join(dir, NECTAR_CONFIG_FILE_NAME), body, "utf8"); @@ -230,6 +230,41 @@ test("AC-018k.9 an unknown key in nectar.json is ignored with a warning; known k } }); +test("a-AC-5 loadNectarFileConfig falls back to legacy nectar.json when the new path is absent", () => { + const root = mkdtempSync(join(tmpdir(), "nectar-cfg-migration-")); + try { + const fleetRoot = join(root, "apiary"); + const runtimeDir = join(fleetRoot, "nectar"); + const legacyDir = join(root, "legacy"); + mkdirSync(runtimeDir, { recursive: true }); + mkdirSync(legacyDir, { recursive: true }); + writeNectarJson(legacyDir, JSON.stringify({ redescribe_threshold: 0.31 })); + + const cfg = loadNectarFileConfig({ env: { APIARY_HOME: fleetRoot }, legacyDir, warn: () => {} }); + assert.equal(cfg.redescribeThreshold, 0.31); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("a-AC-5 loadNectarFileConfig prefers the new nectar.json over legacy when both are present", () => { + const root = mkdtempSync(join(tmpdir(), "nectar-cfg-migration-")); + try { + const fleetRoot = join(root, "apiary"); + const runtimeDir = join(fleetRoot, "nectar"); + const legacyDir = join(root, "legacy"); + mkdirSync(runtimeDir, { recursive: true }); + mkdirSync(legacyDir, { recursive: true }); + writeNectarJson(runtimeDir, JSON.stringify({ redescribe_threshold: 0.82 })); + writeNectarJson(legacyDir, JSON.stringify({ redescribe_threshold: 0.31 })); + + const cfg = loadNectarFileConfig({ env: { APIARY_HOME: fleetRoot }, legacyDir, warn: () => {} }); + assert.equal(cfg.redescribeThreshold, 0.82); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + // ── PRD-018k / NEC-023: brood prerequisite evaluation + guided first-run ──────── test("AC-018k.1/.2 evaluateBroodPrereqs names each missing prerequisite with a machine-readable reason", () => { diff --git a/test/gitignore.test.ts b/test/gitignore.test.ts new file mode 100644 index 0000000..3b7c80c --- /dev/null +++ b/test/gitignore.test.ts @@ -0,0 +1,237 @@ +/** + * PRD-019d: the dependency-free `.gitignore` parser + its wiring into the shared + * ignore predicate and the CLI/walk discovery path. Runs against the compiled + * `dist/` output. Hermetic: the pure-core tests touch no disk; the disk tests + * use temp dirs under `os.tmpdir()` (never the real user home) and clean up. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { compileGitignore, createDiskGitignore } from "../dist/registration/gitignore.js"; +import { + createSharedIgnore, + type GitLsFilesRunner, + type DiskGitignore, +} from "../dist/registration/ignore.js"; +import { createDiskRegistrationFs } from "../dist/registration/disk-fs.js"; +import { discoverFiles } from "../dist/brooding/discovery.js"; + +// ── d-AC-1 / d-AC-3: the pure core ────────────────────────────────────────── + +test("d-AC-1 compileGitignore: a dir-only pattern and a basename glob exclude the right paths; a normal file is included", () => { + const match = compileGitignore(["dist/", "*.log"]); + assert.equal(match("dist/bundle.js"), true, "under dist/ is excluded"); + assert.equal(match("dist", true), true, "the dist directory itself is excluded"); + assert.equal(match("app.log"), true, "*.log is excluded at the root"); + assert.equal(match("logs/app.log"), true, "*.log is a basename glob matched at any depth"); + assert.equal(match("src/x.ts"), false, "a normal source file is included"); +}); + +test("d-AC-3 compileGitignore: a negation re-includes a file an earlier pattern excluded", () => { + const match = compileGitignore(["*.log", "!keep.log"]); + assert.equal(match("keep.log"), false, "keep.log is re-included by the negation"); + assert.equal(match("other.log"), true, "other *.log files stay excluded"); +}); + +// ── Security remediation regression: the collapseGlobStars code path ───────── +// The 2026-07-04 security pass rewrote how `**` runs compile (collapsing +// consecutive `**/` units and `***`+ before compilation) to remove a ReDoS. +// These tests pin (a) that `**` semantics survived the rewrite and (b) that a +// pathological consecutive-globstar pattern stays bounded. + +test("security-regression collapseGlobStars: ** semantics preserved (leading **/, trailing /**, interior a/**/b, consecutive **/**/ collapse)", () => { + // Leading `**/`: matches at any depth, including the root. + const leading = compileGitignore(["**/dist"]); + assert.equal(leading("dist", true), true); + assert.equal(leading("a/dist", true), true); + assert.equal(leading("a/b/dist", true), true); + assert.equal(leading("a/distX", true), false); + + // Trailing `/**`: everything inside the directory. + const trailing = compileGitignore(["a/**"]); + assert.equal(trailing("a/b"), true); + assert.equal(trailing("a/b/c/d"), true); + assert.equal(trailing("b/x"), false); + + // Interior `a/**/b`: zero or more intermediate segments. + const interior = compileGitignore(["a/**/b"]); + assert.equal(interior("a/b"), true, "** matches zero segments"); + assert.equal(interior("a/x/b"), true); + assert.equal(interior("a/x/y/z/b"), true); + assert.equal(interior("a/x/bb"), false); + + // Consecutive `**/**/` collapses to one `**/` with identical semantics. + const consecutive = compileGitignore(["**/**/x"]); + assert.equal(consecutive("x"), true); + assert.equal(consecutive("a/x"), true); + assert.equal(consecutive("a/b/c/x"), true); + assert.equal(consecutive("a/xy"), false); + + // `***`+ is treated as `**`. + const tripleStar = compileGitignore(["a/***"]); + assert.equal(tripleStar("a/b/c"), true); + assert.equal(tripleStar("b/c"), false); +}); + +test("security-regression collapseGlobStars: a pathological consecutive-globstar pattern (20 groups) compiles and matches in bounded time", () => { + // The shape the security report cited: a long `**/**/**/...` chain, which + // pre-fix compiled to a chain of overlapping unbounded quantifiers with + // exponential backtracking against a deep NON-matching path. + const pathological = `${"**/".repeat(20)}*.x`; + const deepNonMatch = `${Array.from({ length: 40 }, (_, i) => `seg${i}`).join("/")}/leaf.txt`; + const deepMatch = `${Array.from({ length: 40 }, (_, i) => `seg${i}`).join("/")}/leaf.x`; + + const startedAt = Date.now(); + const match = compileGitignore([pathological]); + assert.equal(match(deepNonMatch), false, "the non-matching deep path is rejected"); + assert.equal(match(deepMatch), true, "the matching deep path is accepted"); + const elapsedMs = Date.now() - startedAt; + assert.ok(elapsedMs < 250, `compile + both matches must stay bounded; took ${elapsedMs}ms (ceiling 250ms)`); +}); + +test("compileGitignore: anchored (leading-slash / embedded-slash) patterns bind to the root", () => { + const match = compileGitignore(["/build", "a/b"]); + assert.equal(match("build/x"), true, "top-level build is anchored"); + assert.equal(match("pkg/build/x"), false, "a nested build is NOT matched by the anchored /build"); + assert.equal(match("a/b"), true, "an embedded slash anchors the pattern"); + assert.equal(match("z/a/b"), false, "the anchored a/b does not match at a deeper level"); +}); + +// ── d-AC-2: nested .gitignore via the disk loader (injected readFile) ──────── + +test("d-AC-2 createDiskGitignore honors a nested .gitignore relative to its own directory", () => { + const files = new Map([[join("/repo", "packages/a", ".gitignore"), "build/\n"]]); + const match = createDiskGitignore("/repo", { readFile: (p) => files.get(p) ?? null }); + assert.equal(match("packages/a/build/out.js"), true, "packages/a/build is excluded by its own .gitignore"); + assert.equal(match("packages/b/build/out.js"), false, "packages/b has no such rule, so it is included"); +}); + +test("createDiskGitignore: a deeper .gitignore negation overrides a shallower ignore", () => { + const files = new Map([ + [join("/repo", ".gitignore"), "*.log\n"], + [join("/repo", "keep", ".gitignore"), "!*.log\n"], + ]); + const match = createDiskGitignore("/repo", { readFile: (p) => files.get(p) ?? null }); + assert.equal(match("a.log"), true, "root *.log excludes a.log"); + assert.equal(match("keep/a.log"), false, "the deeper !*.log re-includes keep/a.log"); +}); + +test("createDiskGitignore reads .git/info/exclude at the lowest precedence", () => { + const files = new Map([[join("/repo", ".git/info/exclude"), "secret.txt\n"]]); + const match = createDiskGitignore("/repo", { readFile: (p) => files.get(p) ?? null }); + assert.equal(match("secret.txt"), true, "a .git/info/exclude entry is honored"); + assert.equal(match("public.txt"), false); +}); + +// ── d-AC-4 / d-AC-5: the git-present cases (parser is NOT consulted) ───────── + +function gitOk(paths: string[]): GitLsFilesRunner { + return () => ({ status: "ok", paths }); +} +function gitAbsent(): GitLsFilesRunner { + return () => ({ status: "absent" }); +} +function gitError(reason: string): GitLsFilesRunner { + return () => ({ status: "error", reason }); +} + +test("d-AC-4 when git IS available the ls-files snapshot stays authoritative; the gitignore parser is never consulted", () => { + let fallbackCalls = 0; + const fallback: DiskGitignore = () => { + fallbackCalls += 1; + return true; + }; + const shared = createSharedIgnore("/repo", { + readFile: () => null, + gitLsFiles: gitOk(["src/a.ts"]), + gitignoreFallback: fallback, + }); + assert.equal(shared.isIgnored("src/a.ts"), false, "a tracked file is not ignored"); + assert.equal(fallbackCalls, 0, "the parser fallback is never consulted while git is present"); +}); + +test("d-AC-5 git PRESENT but ERRORING keeps the degradation loud and does NOT silently mask it with the parser", () => { + const errors: string[] = []; + let fallbackCalls = 0; + const fallback: DiskGitignore = () => { + fallbackCalls += 1; + return true; + }; + const shared = createSharedIgnore("/repo", { + readFile: () => null, + gitLsFiles: gitError("git ls-files exited with status 128"), + gitignoreFallback: fallback, + onGitError: (reason) => errors.push(reason), + }); + assert.equal(shared.isIgnored("whatever.ts"), false, "no snapshot + errored => not masked by the parser"); + assert.equal(fallbackCalls, 0, "the parser is NOT consulted in the errored case"); + assert.deepEqual(errors, ["git ls-files exited with status 128"], "the degradation is surfaced loudly"); +}); + +test("git GENUINELY ABSENT consults the gitignore parser fallback so a non-git subtree is not gitignore-blind", () => { + let fallbackCalls = 0; + const fallback: DiskGitignore = (rel) => { + fallbackCalls += 1; + return rel === "dist/x.js"; + }; + const shared = createSharedIgnore("/repo", { + readFile: () => null, + gitLsFiles: gitAbsent(), + gitignoreFallback: fallback, + }); + assert.equal(shared.isIgnored("dist/x.js"), true, "the parser verdict is honored when git is absent"); + assert.equal(shared.isIgnored("src/a.ts"), false); + assert.ok(fallbackCalls >= 1, "the parser fallback is consulted when git is genuinely absent"); +}); + +// ── d-AC-6: the walk fallback honors .gitignore on a real non-git root ─────── + +test("d-AC-6 a non-git root: discoverFiles' walk fallback excludes dist/ and *.log via the shared predicate, keeps src/x.ts", () => { + const root = mkdtempSync(join(tmpdir(), "nectar-gitignore-")); + try { + writeFileSync(join(root, ".gitignore"), "dist/\n*.log\n", "utf8"); + mkdirSync(join(root, "src"), { recursive: true }); + mkdirSync(join(root, "dist"), { recursive: true }); + writeFileSync(join(root, "src", "x.ts"), "export const x = 1;\n", "utf8"); + writeFileSync(join(root, "dist", "bundle.js"), "1;\n", "utf8"); + writeFileSync(join(root, "app.log"), "log\n", "utf8"); + + const isIgnored = createSharedIgnore(root).isIgnored; + const fs = createDiskRegistrationFs(root, isIgnored); + // Force the walk fallback (git absent for this temp dir) explicitly. + const result = discoverFiles({ root, fs, isIgnored, gitLsFiles: () => ({ available: false, reason: "absent" }) }); + const paths = result.files.map((f) => f.relPath).sort(); + // `.gitignore` itself is not ignored (git tracks it); dist/ and *.log are excluded. + assert.deepEqual(paths, [".gitignore", "src/x.ts"], "dist/ and *.log are excluded; src/x.ts (and .gitignore) survive"); + assert.ok(!paths.includes("dist/bundle.js"), "dist/ is excluded"); + assert.ok(!paths.includes("app.log"), "*.log is excluded"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +// ── d-AC-7: createDiskRegistrationFs default drops the always-ignored segments ─ + +test("d-AC-7 createDiskRegistrationFs with NO predicate still drops .git / node_modules / .honeycomb (default is createDefaultIgnore, not () => false)", () => { + const ROOT = join("/repo"); + const tree = new Map([ + [ROOT, [dirent("src", "dir"), dirent("node_modules", "dir"), dirent(".git", "dir"), dirent("README.md", "file")]], + [join(ROOT, "src"), [dirent("a.ts", "file")]], + [join(ROOT, "node_modules"), [dirent("pkg", "dir")]], + [join(ROOT, ".git"), [dirent("config", "file")]], + ]); + const fs = createDiskRegistrationFs(ROOT, undefined, (dir) => tree.get(dir) ?? []); + const files = [...fs.listPaths()].sort(); + assert.deepEqual(files, ["README.md", "src/a.ts"], "node_modules and .git are dropped by the default predicate"); +}); + +function dirent(name: string, kind: "file" | "dir"): import("node:fs").Dirent { + return { + name, + isFile: () => kind === "file", + isDirectory: () => kind === "dir", + isSymbolicLink: () => false, + } as import("node:fs").Dirent; +} diff --git a/test/project-context.test.ts b/test/project-context.test.ts new file mode 100644 index 0000000..98ec49e --- /dev/null +++ b/test/project-context.test.ts @@ -0,0 +1,181 @@ +/** + * PRD-019a: the per-project brood + watch RunningContext. Proves a context is + * rooted at ITS OWN bound path and scoped to ITS OWN tenancy, and that start() + * hydrates + watches while stop() drains (a-AC-3 / a-AC-7). Hermetic: a real + * temp dir for the watch root, an injected in-memory store + registration fs + + * ignore (no git spawn, no network). Runs against the compiled `dist/` output. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { createProjectContext } from "../dist/registration/project-context.js"; +import { InMemoryHiveGraphStore } from "../dist/hive-graph/memory-store.js"; +import { sha256Hex } from "../dist/hive-graph/hash.js"; +import { mintNectar } from "../dist/hive-graph/ulid.js"; +import type { AsyncHiveGraphStore } from "../dist/hive-graph/store.js"; +import type { Tenancy } from "../dist/hive-graph/model.js"; + +/** Wrap a sync in-memory store as the async durable seam, counting hydrate reads. */ +function asyncWrap(inner: InMemoryHiveGraphStore, onListLatest: () => void): AsyncHiveGraphStore { + return { + insertIdentity: async (r) => inner.insertIdentity(r), + getIdentity: async (n) => inner.getIdentity(n), + touchIdentity: async (n, d) => inner.touchIdentity(n, d), + appendVersion: async (r) => inner.appendVersion(r), + nextSeq: async (n) => inner.nextSeq(n), + latestVersion: async (n) => inner.latestVersion(n), + listLatestVersions: async (t) => { + onListLatest(); + return inner.listLatestVersions(t); + }, + listLatestDescribedVersions: async (t) => inner.listLatestDescribedVersions(t), + latestVersionByPath: async (t, p) => inner.latestVersionByPath(t, p), + latestVersionByHash: async (t, h) => inner.latestVersionByHash(t, h), + deleteNectar: async (t, n) => inner.deleteNectar(t, n), + }; +} + +const fakeShared = { + isIgnored: (_p: string) => false, + refresh: () => {}, + isGitAvailable: () => false, + lastGitError: () => null, +}; + +/** A registration fs that lists no paths (keeps the resync trivial). */ +const emptyFs = { + statPath: () => null, + existsOnDisk: () => false, + isDirectory: () => false, + listPaths: () => [] as string[], +}; + +test("a-AC-3 / a-AC-7 a project context is rooted at its own path + tenancy; start hydrates + watches, stop drains", async () => { + const root = mkdtempSync(join(tmpdir(), "nectar-ctx-")); + try { + let hydrateReads = 0; + const store = asyncWrap(new InMemoryHiveGraphStore(), () => { + hydrateReads += 1; + }); + const tenancy: Tenancy = { orgId: "o", workspaceId: "w", projectId: "proj-X" }; + const ctx = createProjectContext({ + project: { projectId: "proj-X", path: root, brooding: "active" }, + tenancy, + store, + // no broodDeps -> watch-only (no auto-brood), which keeps this fast + LLM-free + registrationFs: emptyFs as any, + sharedIgnore: fakeShared as any, + } as any); + + assert.equal(ctx.projectId, "proj-X"); + assert.equal(ctx.path, root, "the context is rooted at its OWN bound path"); + assert.equal(ctx.watcherState(), "stopped"); + + await ctx.start(); + assert.equal(ctx.watcherState(), "running", "the watcher is running after start"); + assert.ok(hydrateReads >= 1, "start hydrated the mirror from the durable store (scoped to this tenancy)"); + + await ctx.stop(); + assert.equal(ctx.watcherState(), "stopped", "the watcher is stopped and drained after stop"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("AC-011b.6 (per project) context start loads /.honeycomb/nectars.json and inherits hash-matched files with zero LLM calls", async () => { + const root = mkdtempSync(join(tmpdir(), "nectar-ctx-prewarm-")); + try { + // A real file on disk plus a valid projection whose entry hash-matches it. + const content = "export const x = 1;\n"; + mkdirSync(join(root, "src"), { recursive: true }); + writeFileSync(join(root, "src", "a.ts"), content, "utf8"); + const nectar = mintNectar(); + const projection = { + version: 1, + generated_at: "2026-07-04T00:00:00.000Z", + generator: "honeycomb-nectar@0.0.1", + project: { org_id: "o", workspace_id: "w", project_id: "proj-X" }, + files: { + [nectar]: { + content_hash: sha256Hex(content), + path: "src/a.ts", + title: "The X constant", + description: "Exports x.", + concepts: [], + describe_model: "gemini-2.5-flash", + described_at: "2026-07-04T00:00:00.000Z", + }, + }, + derived: {}, + }; + mkdirSync(join(root, ".honeycomb"), { recursive: true }); + writeFileSync(join(root, ".honeycomb", "nectars.json"), `${JSON.stringify(projection)}\n`, "utf8"); + + const inner = new InMemoryHiveGraphStore(); + const store = asyncWrap(inner, () => {}); + const tenancy: Tenancy = { orgId: "o", workspaceId: "w", projectId: "proj-X" }; + const ctx = createProjectContext({ + project: { projectId: "proj-X", path: root, brooding: "active" }, + tenancy, + store, + // No broodDeps: no describe transport exists, so the ONLY way rows can + // appear is the projection inherit (zero LLM calls by construction). + } as any); + + await ctx.start(); + try { + const identity = inner.getIdentity(nectar); + assert.ok(identity !== undefined, "the projection's nectar identity was inherited into the store"); + const latest = inner.latestVersion(nectar); + assert.equal(latest?.path, "src/a.ts"); + assert.equal(latest?.title, "The X constant"); + assert.equal(latest?.description, "Exports x."); + assert.equal(latest?.contentHash, sha256Hex(content), "the inherited row carries the hash-matched content"); + } finally { + await ctx.stop(); + } + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("AC-011b.6 (per project) a missing or foreign-tenancy projection is skipped fail-soft; the context still starts", async () => { + const root = mkdtempSync(join(tmpdir(), "nectar-ctx-prewarm-skip-")); + try { + // A projection for a DIFFERENT project triple: validation rejects it whole. + mkdirSync(join(root, ".honeycomb"), { recursive: true }); + writeFileSync( + join(root, ".honeycomb", "nectars.json"), + JSON.stringify({ + version: 1, + generated_at: "2026-07-04T00:00:00.000Z", + generator: "g", + project: { org_id: "OTHER", workspace_id: "w", project_id: "proj-Y" }, + files: {}, + derived: {}, + }), + "utf8", + ); + const inner = new InMemoryHiveGraphStore(); + const store = asyncWrap(inner, () => {}); + const tenancy: Tenancy = { orgId: "o", workspaceId: "w", projectId: "proj-X" }; + const ctx = createProjectContext({ + project: { projectId: "proj-X", path: root, brooding: "active" }, + tenancy, + store, + registrationFs: emptyFs as any, + sharedIgnore: fakeShared as any, + } as any); + await ctx.start(); + try { + assert.equal(inner.listLatestVersions(tenancy).length, 0, "nothing is inherited from a foreign projection"); + assert.equal(ctx.watcherState(), "running", "the context still starts (fail-soft skip)"); + } finally { + await ctx.stop(); + } + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); diff --git a/test/project-supervisor.test.ts b/test/project-supervisor.test.ts new file mode 100644 index 0000000..23a0c14 --- /dev/null +++ b/test/project-supervisor.test.ts @@ -0,0 +1,221 @@ +/** + * PRD-019a: the multi-root ProjectSupervisor + the ActiveProjectsController + * reconcile driver, exercised with FAKE contexts so the start/stop/drain and + * reconcile-on-change behavior is deterministic and hermetic (no disk, no + * network). Runs against the compiled `dist/` output. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ProjectSupervisor } from "../dist/project-supervisor.js"; +import { ActiveProjectsController } from "../dist/active-projects-runtime.js"; +import { createProjectContext } from "../dist/registration/project-context.js"; +import { InMemoryHiveGraphStore } from "../dist/hive-graph/memory-store.js"; +import { readActiveProjects, persistProjectBrooding } from "../dist/projects-control.js"; +import type { AsyncHiveGraphStore } from "../dist/hive-graph/store.js"; + +type Ev = string; + +function fakeContext(events: Ev[], projectId: string, path: string, opts: { failStop?: boolean } = {}) { + let state: "stopped" | "running" = "stopped"; + return { + projectId, + path, + watcherState: () => (state === "running" ? "running" : "stopped"), + async start() { + events.push(`start:${projectId}`); + state = "running"; + }, + async stop() { + events.push(`stop:${projectId}`); + state = "stopped"; + if (opts.failStop === true) throw new Error("boom"); + }, + } as const; +} + +test("a-AC-3/a-AC-4 reconcile starts one context per active project", async () => { + const events: Ev[] = []; + const sup = new ProjectSupervisor({ + factory: (p) => fakeContext(events, p.projectId, p.path), + }); + await sup.reconcile([ + { projectId: "a", path: "/a", brooding: "active" }, + { projectId: "b", path: "/b", brooding: "active" }, + ]); + assert.deepEqual(events.sort(), ["start:a", "start:b"]); + assert.equal(sup.contexts().length, 2); + assert.equal(sup.watcherStateFor("a"), "running"); + assert.equal(sup.watcherStateFor("missing"), "stopped"); +}); + +test("a-AC-5 reconcile starts a newly-bound project and stops an unbound one with no restart", async () => { + const events: Ev[] = []; + const sup = new ProjectSupervisor({ factory: (p) => fakeContext(events, p.projectId, p.path) }); + await sup.reconcile([{ projectId: "a", path: "/a", brooding: "active" }]); + events.length = 0; + // 'a' unbound, 'b' newly bound. + await sup.reconcile([{ projectId: "b", path: "/b", brooding: "active" }]); + assert.deepEqual(events.sort(), ["start:b", "stop:a"]); + assert.deepEqual(sup.contexts().map((c) => c.projectId), ["b"]); +}); + +test("reconcile restarts a context whose bound path changed", async () => { + const events: Ev[] = []; + const sup = new ProjectSupervisor({ factory: (p) => fakeContext(events, p.projectId, p.path) }); + await sup.reconcile([{ projectId: "a", path: "/old", brooding: "active" }]); + events.length = 0; + await sup.reconcile([{ projectId: "a", path: "/new", brooding: "active" }]); + assert.deepEqual(events, ["stop:a", "start:a"], "a path change stops the old and starts the new"); + assert.equal(sup.get("a")?.path, "/new"); +}); + +test("a-AC-7 stopAll stops every running context (drain on teardown); a failing stop is isolated, not thrown", async () => { + const events: Ev[] = []; + const errors: Array<[string, string]> = []; + const sup = new ProjectSupervisor({ + factory: (p) => fakeContext(events, p.projectId, p.path, { failStop: p.projectId === "b" }), + onError: (scope, id) => errors.push([scope, id]), + }); + await sup.reconcile([ + { projectId: "a", path: "/a", brooding: "active" }, + { projectId: "b", path: "/b", brooding: "active" }, + ]); + events.length = 0; + await sup.stopAll(); + assert.deepEqual(events.sort(), ["stop:a", "stop:b"]); + assert.equal(sup.contexts().length, 0); + assert.deepEqual(errors, [["stop", "b"]], "the failing stop routes to onError, never throws out of reconcile"); +}); + +test("reconciles are serialized: two concurrent reconciles never double-start a context", async () => { + const events: Ev[] = []; + const sup = new ProjectSupervisor({ factory: (p) => fakeContext(events, p.projectId, p.path) }); + const active = [{ projectId: "a", path: "/a", brooding: "active" as const }]; + await Promise.all([sup.reconcile(active), sup.reconcile(active)]); + assert.deepEqual( + events.filter((e) => e === "start:a"), + ["start:a"], + "'a' is started exactly once despite two overlapping reconciles", + ); +}); + +// ── b-AC-4: OFF then ON resumes watch + brood with a cold-catch-up resync ──── + +test("b-AC-4 a paused project set back to on: the reconcile resumes its watch + brood and a cold-catch-up resync runs (019a start path)", async () => { + const tempRoot = mkdtempSync(join(tmpdir(), "nectar-bac4-")); + const projectRoot = join(tempRoot, "repo"); + const stateDir = join(tempRoot, "state"); + const cacheDir = join(tempRoot, "deeplake"); + try { + mkdirSync(projectRoot, { recursive: true }); + mkdirSync(cacheDir, { recursive: true }); + // One bound project in a hermetic (temp) shared-cache dir. + writeFileSync( + join(cacheDir, "projects.json"), + JSON.stringify({ + schemaVersion: 1, + org: "o", + workspace: "w", + bindings: [{ path: projectRoot, projectId: "p1" }], + projects: [], + }), + "utf8", + ); + const controlOptions = { cacheDir, broodingState: { dir: stateDir } } as const; + + // A REAL project context (the 019a start path): its start() hydrates, + // starts the watcher, and requests the cold-catch-up resync, which funnels + // through the shared-ignore refresh seam counted here. + let resyncs = 0; + const sharedIgnore = { + isIgnored: () => false, + refresh: () => { + resyncs += 1; + }, + isGitAvailable: () => false, + lastGitError: () => null, + }; + const emptyFs = { statPath: () => null, existsOnDisk: () => false, isDirectory: () => false, listPaths: () => [] }; + const inner = new InMemoryHiveGraphStore(); + const store: AsyncHiveGraphStore = { + insertIdentity: async (r) => inner.insertIdentity(r), + getIdentity: async (n) => inner.getIdentity(n), + touchIdentity: async (n, d) => inner.touchIdentity(n, d), + appendVersion: async (r) => inner.appendVersion(r), + nextSeq: async (n) => inner.nextSeq(n), + latestVersion: async (n) => inner.latestVersion(n), + listLatestVersions: async (t) => inner.listLatestVersions(t), + listLatestDescribedVersions: async (t) => inner.listLatestDescribedVersions(t), + latestVersionByPath: async (t, p) => inner.latestVersionByPath(t, p), + latestVersionByHash: async (t, h) => inner.latestVersionByHash(t, h), + deleteNectar: async (t, n) => inner.deleteNectar(t, n), + }; + + const controller = new ActiveProjectsController({ + resolve: () => readActiveProjects(controlOptions).resolution, + factory: (project) => + createProjectContext({ + project, + tenancy: { orgId: "o", workspaceId: "w", projectId: project.projectId }, + store, + registrationFs: emptyFs as any, + sharedIgnore: sharedIgnore as any, + } as any), + setHealth: () => {}, + intervalMs: 60_000, + }); + try { + // 1. Bound + brooding-on by default: the first reconcile starts the + // context (watch running, one cold-catch-up resync). + await controller.reconcileNow(); + assert.equal(controller.supervisor.watcherStateFor("p1"), "running"); + assert.equal(resyncs, 1, "the initial start ran its cold-catch-up resync"); + + // 2. Pause it (the persisted OFF the toggle API writes): the reconcile + // stops its watch + brood. + persistProjectBrooding("p1", "off", controlOptions); + await controller.reconcileNow(); + assert.equal(controller.supervisor.watcherStateFor("p1"), "stopped", "OFF stopped the watch"); + + // 3. b-AC-4: set it back to ON. The reconcile resumes watch + brood via + // the 019a start path, and a FRESH cold-catch-up resync runs. + persistProjectBrooding("p1", "on", controlOptions); + await controller.reconcileNow(); + assert.equal(controller.supervisor.watcherStateFor("p1"), "running", "ON resumed the watch"); + assert.equal(resyncs, 2, "the resume ran a fresh cold-catch-up resync"); + } finally { + await controller.stop(); + } + } finally { + rmSync(tempRoot, { recursive: true, force: true }); + } +}); + +// ── ActiveProjectsController: resolve + reconcile + health publish ─────────── + +test("ActiveProjectsController.reconcileNow resolves, reconciles, and publishes the /health slice", async () => { + const events: Ev[] = []; + const slices: Array<{ count: number; reason: string | null }> = []; + let active = [{ projectId: "a", path: "/a", brooding: "active" as const }]; + const controller = new ActiveProjectsController({ + resolve: () => ({ projects: active, refused: [], active, globalPaused: false }), + factory: (p) => fakeContext(events, p.projectId, p.path), + setHealth: (slice) => slices.push({ count: slice.count, reason: slice.reason }), + intervalMs: 1000, + }); + await controller.reconcileNow(); + assert.deepEqual(events, ["start:a"]); + assert.equal(slices.at(-1)?.count, 1); + assert.equal(slices.at(-1)?.reason, null); + + // Empty the active set; a reconcile stops the context and health goes dormant. + active = []; + await controller.reconcileNow(); + assert.deepEqual(events, ["start:a", "stop:a"]); + assert.equal(slices.at(-1)?.count, 0); + assert.equal(slices.at(-1)?.reason, "no-active-projects"); + await controller.stop(); +}); diff --git a/test/projects-api.test.ts b/test/projects-api.test.ts new file mode 100644 index 0000000..3b59406 --- /dev/null +++ b/test/projects-api.test.ts @@ -0,0 +1,148 @@ +/** + * PRD-019b: the projects + brooding-control endpoints, exercised through the + * router seam with injected mechanics (the daemon side the hive dashboard calls + * is real). Also covers the hand-validated toggle body and the persist/reconcile + * ordering (b-AC-3/4/5/6). Imports the compiled modules from `dist/`. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { NectarRouter, type RouteContext, type RouteResponse } from "../dist/api/router.js"; +import { mountProjectsApi, parseBroodingToggle, type MountProjectsOptions } from "../dist/api/projects-api.js"; +import type { ProjectsView } from "../dist/projects-control.js"; + +function makeCtx(method: string, path: string, opts: { body?: unknown } = {}): RouteContext { + return { + method, + path, + rawUrl: path, + query: new URLSearchParams(""), + headers: {}, + body: () => opts.body, + json: (body: unknown, status = 200): RouteResponse => ({ + status, + body: JSON.stringify(body ?? null), + contentType: "application/json; charset=utf-8", + }), + }; +} + +function mount(options: MountProjectsOptions): NectarRouter { + const router = new NectarRouter(); + mountProjectsApi({ group: (p) => router.group(p) }, options); + return router; +} + +async function call(router: NectarRouter, method: string, path: string, opts: { body?: unknown } = {}) { + const res = await router.dispatch(makeCtx(method, path, opts)); + assert.ok(res !== undefined, `${method} ${path} should be handled`); + return { status: res!.status, body: JSON.parse(res!.body) }; +} + +const VIEW: ProjectsView = { + globalBrooding: "on", + projects: [ + { projectId: "p1", name: "Repo One", path: "/work/one", brooding: "active", watcher: "running", counts: null }, + ], +}; + +test("GET /api/hive-graph/projects returns the view shape the dashboard consumes", async () => { + const router = mount({ + view: () => VIEW, + setProject: () => {}, + setGlobal: () => {}, + reconcile: async () => {}, + }); + const res = await call(router, "GET", "/api/hive-graph/projects"); + assert.equal(res.status, 200); + assert.deepEqual(res.body, VIEW, "GET returns { globalBrooding, projects: [{ projectId, name, path, brooding, watcher, counts }] }"); +}); + +test("b-AC-3 POST { projectId, brooding: off } persists then reconciles, returning the new view", async () => { + const calls: string[] = []; + const router = mount({ + view: () => VIEW, + setProject: (id, b) => calls.push(`set:${id}:${b}`), + setGlobal: () => calls.push("global"), + reconcile: async () => calls.push("reconcile"), + }); + const res = await call(router, "POST", "/api/hive-graph/projects/brooding", { body: { projectId: "p1", brooding: "off" } }); + assert.equal(res.status, 200); + assert.deepEqual(calls, ["set:p1:off", "reconcile"], "persist happens BEFORE reconcile"); + assert.deepEqual(res.body, VIEW); +}); + +test("b-AC-5 POST { global: paused } sets the global switch then reconciles", async () => { + const calls: string[] = []; + const router = mount({ + view: () => VIEW, + setProject: () => calls.push("project"), + setGlobal: (g) => calls.push(`global:${g}`), + reconcile: async () => calls.push("reconcile"), + }); + const res = await call(router, "POST", "/api/hive-graph/projects/brooding", { body: { global: "paused" } }); + assert.equal(res.status, 200); + assert.deepEqual(calls, ["global:paused", "reconcile"]); +}); + +test("b-AC-6 a persist (disk) failure returns 500 with a REDACTED reason and does NOT reconcile (prior state intact)", async () => { + const calls: string[] = []; + const observed: unknown[] = []; + const rawError = new Error("EACCES: permission denied, open 'C:\\Users\\op\\.apiary\\nectar\\.projects.json.123.456.tmp'"); + const router = mount({ + view: () => VIEW, + setProject: () => { + throw rawError; + }, + setGlobal: () => {}, + reconcile: async () => calls.push("reconcile"), + stateFilePath: "/state/projects.json", + onPersistError: (err) => observed.push(err), + }); + const res = await call(router, "POST", "/api/hive-graph/projects/brooding", { body: { projectId: "p1", brooding: "off" } }); + assert.equal(res.status, 500); + assert.equal(res.body.error, "persist_failed"); + // Redaction: the stable reason names only the state file, never the raw OS error. + assert.equal(res.body.reason, "could not persist the brooding state (/state/projects.json)"); + assert.ok(!/EACCES|errno|permission denied|\.tmp/.test(JSON.stringify(res.body)), "no raw errno text leaks into the body"); + // The raw failure still reaches the server-side observer (daemon log), so diagnostics are not lost. + assert.deepEqual(observed, [rawError]); + assert.deepEqual(calls, [], "the reconcile is NOT run against a half-written file"); +}); + +test("b-AC-6 a persist failure with no stateFilePath configured still redacts to the stable reason only", async () => { + const router = mount({ + view: () => VIEW, + setProject: () => { + throw new Error("ENOSPC: no space left on device, write"); + }, + setGlobal: () => {}, + reconcile: async () => {}, + }); + const res = await call(router, "POST", "/api/hive-graph/projects/brooding", { body: { projectId: "p1", brooding: "off" } }); + assert.equal(res.status, 500); + assert.equal(res.body.reason, "could not persist the brooding state"); + assert.ok(!/ENOSPC/.test(JSON.stringify(res.body))); +}); + +test("an invalid toggle body is a 400 (never a 500) and never persists", async () => { + const calls: string[] = []; + const router = mount({ + view: () => VIEW, + setProject: () => calls.push("set"), + setGlobal: () => calls.push("set"), + reconcile: async () => calls.push("reconcile"), + }); + const bad = await call(router, "POST", "/api/hive-graph/projects/brooding", { body: { projectId: "p1", brooding: "maybe" } }); + assert.equal(bad.status, 400); + assert.equal(bad.body.error, "invalid_request"); + assert.deepEqual(calls, [], "nothing is persisted for an invalid body"); +}); + +test("parseBroodingToggle accepts exactly one of { projectId, brooding } or { global } and rejects the rest", () => { + assert.deepEqual(parseBroodingToggle({ projectId: "p", brooding: "on" }), { kind: "project", projectId: "p", brooding: "on" }); + assert.deepEqual(parseBroodingToggle({ global: "paused" }), { kind: "global", global: "paused" }); + assert.throws(() => parseBroodingToggle({ projectId: "p", brooding: "on", global: "on" }), /not both/); + assert.throws(() => parseBroodingToggle({ brooding: "on" }), /projectId/); + assert.throws(() => parseBroodingToggle({ global: "nope" }), /"on" or "paused"/); + assert.throws(() => parseBroodingToggle(42), /object/); +}); diff --git a/test/projects-cli.test.ts b/test/projects-cli.test.ts new file mode 100644 index 0000000..45604b3 --- /dev/null +++ b/test/projects-cli.test.ts @@ -0,0 +1,54 @@ +/** + * PRD-019b: the `nectar projects` / `nectar brooding` CLI grammar + rendering. + * The parser is a pure function (hermetic); the write side is a thin loopback + * client of the same POST endpoint the dashboard uses, so the parse coverage + * plus the projects-api coverage prove b-AC-7 (CLI == API effect). Imports the + * compiled modules from `dist/`. + */ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { parseBroodingArgs, renderProjectsTable } from "../dist/cli.js"; +import type { ProjectsView } from "../dist/projects-control.js"; + +test("b-AC-7 parseBroodingArgs: `on/off --project ` targets one project", () => { + assert.deepEqual(parseBroodingArgs(["off", "--project", "p1"]), { kind: "project", projectId: "p1", brooding: "off" }); + assert.deepEqual(parseBroodingArgs(["on", "--project=p2"]), { kind: "project", projectId: "p2", brooding: "on" }); +}); + +test("parseBroodingArgs: `on/off --all` targets every bound project", () => { + assert.deepEqual(parseBroodingArgs(["on", "--all"]), { kind: "all", brooding: "on" }); +}); + +test("parseBroodingArgs: the global flags flip the global switch and take no on|off argument", () => { + assert.deepEqual(parseBroodingArgs(["--global-pause"]), { kind: "global", global: "paused" }); + assert.deepEqual(parseBroodingArgs(["--global-resume"]), { kind: "global", global: "on" }); +}); + +test("parseBroodingArgs: malformed invocations are reported as errors, never a silent default", () => { + assert.equal(parseBroodingArgs(["on"]).kind, "errors", "missing --project/--all is an error"); + assert.equal(parseBroodingArgs(["maybe", "--all"]).kind, "errors", "a non on|off positional is an error"); + assert.equal(parseBroodingArgs(["on", "--project", "p", "--all"]).kind, "errors", "project + all is an error"); + assert.equal(parseBroodingArgs(["on", "--global-pause"]).kind, "errors", "global flag + on|off is an error"); + assert.equal(parseBroodingArgs(["--global-pause", "--global-resume"]).kind, "errors", "two global flags is an error"); + assert.equal(parseBroodingArgs(["off", "--nope"]).kind, "errors", "an unknown flag is an error"); +}); + +test("renderProjectsTable renders the global switch and each project's brooding + watcher state", () => { + const view: ProjectsView = { + globalBrooding: "on", + projects: [ + { projectId: "p1", name: "Repo One", path: "/work/one", brooding: "active", watcher: "running", counts: null }, + { projectId: "p2", name: "", path: "/work/two", brooding: "paused", watcher: "stopped", counts: null }, + ], + }; + const out = renderProjectsTable(view); + assert.match(out, /global brooding: on/); + assert.match(out, /Repo One \(p1\)/); + assert.match(out, /brooding: active {3}watcher: running/); + assert.match(out, /brooding: paused {3}watcher: stopped/); +}); + +test("renderProjectsTable shows the empty-state guidance when there are no active projects", () => { + const out = renderProjectsTable({ globalBrooding: "on", projects: [] }); + assert.match(out, /No active projects/); +}); diff --git a/test/service-argv.test.ts b/test/service-argv.test.ts index 8b9a25c..0b442e5 100644 --- a/test/service-argv.test.ts +++ b/test/service-argv.test.ts @@ -74,6 +74,18 @@ test("sc (Windows system-scope) create + start; stop + delete; query", () => { assert.deepEqual(statusCommand(p, 0).args, ["query", "nectar"]); }); +test("c-AC-5 sc binPath carries APIARY_HOME when the plan pins a root", () => { + const p = plan({ + platform: "win32", + home: "C:/Users/op", + privileged: true, + preferSystemScope: true, + apiaryHome: "C:/Pinned/Home", + }); + const create = installCommands(p, 0)[0]; + assert.ok(create?.args.some((arg) => arg.includes("APIARY_HOME=C:/Pinned/Home"))); +}); + test("legacy dereg (decision #32 migration) targets the pre-rename unit names on every platform", () => { const mac = plan({ platform: "darwin", home: "/Users/op" }); assert.deepEqual(legacyUninstallCommands(mac, 501)[0]?.args, ["bootout", "gui/501/com.hivenectar.daemon"]); diff --git a/test/service-index.test.ts b/test/service-index.test.ts index 373d6e7..468dfc5 100644 --- a/test/service-index.test.ts +++ b/test/service-index.test.ts @@ -183,6 +183,23 @@ test("install on darwin writes the launchd plist to ~/Library/LaunchAgents and b assert.equal(calls[2]?.args[0], "kickstart"); }); +test("c-AC-2 install renders APIARY_HOME into launchd when the plan pins a custom root", async () => { + const fs = fakeFs(); + const svc = createServiceModule({ + execPath: "/usr/local/bin/nectar", + fs, + runner: okRunner(), + environment: linuxEnv({ platform: "darwin", home: "/Users/op", apiaryHome: "/custom/root" }), + }); + const result = await svc.install(); + assert.equal(result.ok, true); + const unitPath = "/Users/op/Library/LaunchAgents/com.legioncode.nectar.plist"; + const plist = fs.written.get(unitPath) ?? ""; + assert.match(plist, /APIARY_HOME<\/key>/); + assert.match(plist, /\/custom\/root<\/string>/); + assert.ok(fs.dirs.includes("/custom/root/nectar/logs"), "launchd log dir follows the pinned root"); +}); + test("AC-018l.9 the macOS (launchd) install creates the log directory the plist writes to (NEC-042 item 2)", async () => { const fs = fakeFs(); const svc = createServiceModule({ @@ -193,9 +210,9 @@ test("AC-018l.9 the macOS (launchd) install creates the log directory the plist }); const result = await svc.install(); assert.equal(result.ok, true); - // The plist writes stdout/stderr to /.honeycomb/nectar; that dir must exist. + // The plist writes stdout/stderr to /.apiary/nectar/logs; that dir must exist. assert.ok( - fs.dirs.includes("/Users/op/.honeycomb/nectar"), + fs.dirs.includes("/Users/op/.apiary/nectar/logs"), `the launchd log directory was mkdirp'd (saw: ${JSON.stringify(fs.dirs)})`, ); }); @@ -228,7 +245,7 @@ test("install on win32 stages the schtasks XML beside the workspace, then Create assert.equal(result.ok, true); assert.match(result.message, /registered as a schtasks service/); - const stagedXml = "C:/Users/op/.honeycomb/nectar/nectar-task.xml"; + const stagedXml = "C:/Users/op/.apiary/nectar/nectar-task.xml"; assert.ok(fs.written.has(stagedXml), "the schtasks XML is staged beside the workspace (unitPath is empty for schtasks)"); assert.match(fs.written.get(stagedXml) ?? "", / { + const fs = fakeFs(); + const calls: { command: string; args: readonly string[] }[] = []; + const svc = createServiceModule({ + execPath: "C:/Program Files/nectar/nectar.js", + fs, + runner: okRunner(calls), + environment: linuxEnv({ + platform: "win32", + home: "C:/Users/op", + privileged: true, + preferSystemScope: true, + apiaryHome: "C:/Pinned/Home", + }), + }); + const result = await svc.install(); + assert.equal(result.ok, true); + const create = calls.find((call) => call.command === "sc" && call.args[0] === "create"); + assert.ok(create, "sc create command was issued"); + assert.ok(create?.args.some((arg) => arg.includes("APIARY_HOME=C:/Pinned/Home"))); }); test("serviceStatus classifies systemd is-active output and never throws on an unsupported platform", async () => { diff --git a/test/service-platform.test.ts b/test/service-platform.test.ts index 14eea98..cd3b694 100644 --- a/test/service-platform.test.ts +++ b/test/service-platform.test.ts @@ -65,6 +65,11 @@ test("win32 system scope (privileged + requested) uses sc, not schtasks", () => assert.equal(plan.scope, "system"); }); +test("c-AC-2 resolveServicePlan carries a pinned APIARY_HOME into the plan", () => { + const plan = resolveServicePlan(env({ platform: "darwin", apiaryHome: "/custom/root" })); + assert.equal(plan.apiaryHome, "/custom/root"); +}); + test("an unsupported platform throws rather than silently degrading", () => { assert.throws(() => resolveServicePlan(env({ platform: "aix" as NodeJS.Platform })), /unsupported platform/); assert.equal(normalizePlatform("aix" as NodeJS.Platform), null); diff --git a/test/service-templates.test.ts b/test/service-templates.test.ts index 7e0ec03..36ed4da 100644 --- a/test/service-templates.test.ts +++ b/test/service-templates.test.ts @@ -11,6 +11,8 @@ import { quoteSystemdToken, escapeXml, } from "../dist/service/templates.js"; +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; import { resolveServicePlan } from "../dist/service/platform.js"; function plan(overrides: Partial[0]> = {}) { @@ -34,6 +36,20 @@ test("launchd plist encodes RunAtLoad + KeepAlive + the daemon run command", () assert.match(xml, new RegExp(`${RESTART_SEC}`)); assert.match(xml, /com\.legioncode\.nectar<\/string>/); assert.match(xml, /daemon<\/string>/); + assert.match(xml, /\/Users\/op\/\.apiary\/nectar\/logs\/launchd\.out\.log<\/string>/); +}); + +test("c-AC-1 service templates carry no .honeycomb literal", () => { + const src = readFileSync(fileURLToPath(new URL("../dist/service/templates.js", import.meta.url)), "utf8"); + assert.doesNotMatch(src, /\.honeycomb/); +}); + +test("c-AC-2 launchd plist includes APIARY_HOME when the plan pins a custom root", () => { + const xml = renderLaunchdPlist(plan({ platform: "darwin", home: "/Users/op", apiaryHome: "/custom/root" })); + assert.match(xml, /EnvironmentVariables<\/key>/); + assert.match(xml, /APIARY_HOME<\/key>/); + assert.match(xml, /\/custom\/root<\/string>/); + assert.match(xml, /\/custom\/root\/nectar\/logs\/launchd\.out\.log<\/string>/); }); test("systemd unit encodes Restart=always + RestartSec + WantedBy + the daemon run command, prefixed by the node interpreter", () => { @@ -50,6 +66,11 @@ test("systemd unit encodes Restart=always + RestartSec + WantedBy + the daemon r ); }); +test("c-AC-2 systemd unit pins APIARY_HOME when the plan carries a custom fleet root", () => { + const unit = renderSystemdUnit(plan({ platform: "linux", apiaryHome: "/custom/root" })); + assert.match(unit, /Environment="APIARY_HOME=\/custom\/root"/); +}); + test("AC-018a.8 systemd unit declares finite restart rate limiting (no StartLimitIntervalSec=0 crash loop)", () => { const unit = renderSystemdUnit(plan({ platform: "linux" })); assert.match(unit, /StartLimitBurst=\d+/, "a finite StartLimitBurst is declared"); @@ -68,7 +89,13 @@ test("Windows Scheduled Task XML encodes LogonTrigger + RestartOnFailure at the assert.match(xml, new RegExp(`${WINDOWS_RESTART_INTERVAL}`)); assert.match(xml, /IgnoreNew<\/MultipleInstancesPolicy>/); assert.match(xml, /\\nectar<\/URI>/); - assert.match(xml, /Arguments>".*nectar" daemon[\s\S]*daemon<\/Arguments>/); +}); + +test("c-AC-5 Windows task XML carries APIARY_HOME through cmd when the plan pins a root", () => { + const xml = renderScheduledTaskXml(plan({ platform: "win32", home: "C:/Users/op", apiaryHome: "C:/Pinned/Home" })); + assert.match(xml, /cmd\.exe<\/Command>/); + assert.match(xml, /APIARY_HOME=C:\/Pinned\/Home/); }); test("renderUnit dispatches per manager", () => { diff --git a/test/state-migration.test.ts b/test/state-migration.test.ts new file mode 100644 index 0000000..10106b9 --- /dev/null +++ b/test/state-migration.test.ts @@ -0,0 +1,391 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { + MIGRATION_MARKER_FILE_NAME, + assertNoLegacyDaemonRunning, + resolveStateReadPath, + runStateMigration, +} from "../dist/state-migration.js"; +import { DoctorRegistryError, registerWithDoctor } from "../dist/doctor-registry.js"; +import { assembleDaemon } from "../dist/index.js"; + +function tmpPaths() { + const root = mkdtempSync(join(tmpdir(), "nectar-state-migration-")); + const legacyDir = join(root, "legacy"); + const fleetRoot = join(root, "apiary"); + const runtimeDir = join(fleetRoot, "nectar"); + return { root, legacyDir, fleetRoot, runtimeDir }; +} + +function writeFile(path: string, body: string): void { + mkdirSync(dirname(path), { recursive: true }); + writeFileSync(path, body, "utf8"); +} + +const baseConfig = (runtimeDir: string) => ({ + runtimeDir, + host: "127.0.0.1", + port: 3854, + pidFilePath: join(runtimeDir, "nectar.pid"), +}); + +test("b-AC-1 migrates allow-listed files into /nectar and writes a marker", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "nectar.json"), '{"k":"v"}\n'); + writeFile(join(t.legacyDir, "pending-reviews.json"), '["a"]\n'); + writeFile(join(t.legacyDir, "telemetry", "nectar.sqlite"), "sqlite"); + writeFile(join(t.legacyDir, "nectar-usage-telemetry.json"), '{"reported":[]}\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + + const result = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + nowIso: () => "2026-07-04T00:00:00.000Z", + }); + + assert.equal(result.failed.length, 0); + assert.deepEqual(result.moved.sort(), [ + "nectar-usage-telemetry.json", + "nectar.json", + "pending-reviews.json", + "telemetry/nectar.sqlite", + ]); + assert.equal(existsSync(join(t.runtimeDir, "nectar.json")), true); + assert.equal(existsSync(join(t.runtimeDir, "pending-reviews.json")), true); + assert.equal(existsSync(join(t.runtimeDir, "telemetry", "nectar.sqlite")), true); + assert.equal(existsSync(join(t.runtimeDir, "nectar-usage-telemetry.json")), true); + assert.equal(existsSync(join(t.legacyDir, "nectar.json")), false); + assert.equal(existsSync(join(t.legacyDir, "pending-reviews.json")), false); + assert.equal(existsSync(join(t.legacyDir, "telemetry", "nectar.sqlite")), false); + assert.equal(existsSync(join(t.legacyDir, "nectar-usage-telemetry.json")), false); + assert.equal(existsSync(join(t.runtimeDir, MIGRATION_MARKER_FILE_NAME)), true); + assert.equal(existsSync(join(t.fleetRoot, "registry.json")), true); + const registry = JSON.parse(readFileSync(join(t.fleetRoot, "registry.json"), "utf8")); + const nectarEntry = (registry.daemons as Array<{ name: string; pidPath: string; telemetryDbPath: string }>).find( + (entry) => entry.name === "nectar", + ); + assert.equal(nectarEntry?.pidPath, join(t.runtimeDir, "nectar.pid")); + assert.equal(nectarEntry?.telemetryDbPath, join(t.runtimeDir, "telemetry", "nectar.sqlite")); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("b-AC-2 rerunning migration after completion is idempotent (no additional moves)", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "nectar.json"), '{"k":"v"}\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + const second = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + assert.equal(second.skipped, true); + assert.deepEqual(second.moved, []); + assert.deepEqual(second.failed, []); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("b-AC-3 failed file migration leaves legacy source intact and retries successfully on next boot", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "pending-reviews.json"), '["legacy"]\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + const failed = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + migrateFile: (sourcePath, targetPath) => { + if (targetPath.endsWith("pending-reviews.json")) throw new Error("simulated copy failure"); + writeFile(targetPath, readFileSync(sourcePath, "utf8")); + }, + }); + assert.deepEqual(failed.failed, ["pending-reviews.json"]); + assert.equal(existsSync(join(t.legacyDir, "pending-reviews.json")), true); + assert.equal(existsSync(join(t.runtimeDir, "pending-reviews.json")), false); + + const retried = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + assert.deepEqual(retried.failed, []); + assert.ok(retried.moved.includes("pending-reviews.json")); + assert.equal(existsSync(join(t.runtimeDir, "pending-reviews.json")), true); + assert.equal(existsSync(join(t.legacyDir, "pending-reviews.json")), false); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("b-AC-4 legacy live pid blocks startup; stale legacy pid allows startup guard to pass", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "nectar.pid"), String(process.pid)); + writeFile(join(t.legacyDir, "nectar.lock"), String(process.pid)); + assert.throws( + () => + assertNoLegacyDaemonRunning({ + runtimeDir: t.runtimeDir, + pidFilePath: join(t.runtimeDir, "nectar.pid"), + lockFilePath: join(t.runtimeDir, "nectar.lock"), + legacyDir: t.legacyDir, + }), + /already running/i, + ); + + writeFile(join(t.legacyDir, "nectar.pid"), "2147483600"); + assert.doesNotThrow(() => + assertNoLegacyDaemonRunning({ + runtimeDir: t.runtimeDir, + pidFilePath: join(t.runtimeDir, "nectar.pid"), + lockFilePath: join(t.runtimeDir, "nectar.lock"), + legacyDir: t.legacyDir, + }), + ); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("b-AC-5 resolveStateReadPath prefers new path and falls back to legacy when new path is absent", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "pending-reviews.json"), '["legacy"]'); + const fallback = resolveStateReadPath("pending-reviews.json", { runtimeDir: t.runtimeDir, legacyDir: t.legacyDir }); + assert.equal(fallback, join(t.legacyDir, "pending-reviews.json")); + + writeFile(join(t.runtimeDir, "pending-reviews.json"), '["new"]'); + const preferred = resolveStateReadPath("pending-reviews.json", { runtimeDir: t.runtimeDir, legacyDir: t.legacyDir }); + assert.equal(preferred, join(t.runtimeDir, "pending-reviews.json")); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("b-AC-6 migration allow-list leaves non-nectar legacy files byte-identical", () => { + const t = tmpPaths(); + try { + const nonOwnedPath = join(t.legacyDir, "doctor.daemons.json"); + writeFile(nonOwnedPath, '{"daemons":[]}\n'); + writeFile(join(t.legacyDir, "nectar.json"), '{"k":"v"}\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + + runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + + assert.equal(readFileSync(nonOwnedPath, "utf8"), '{"daemons":[]}\n'); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("b-AC-7 partially migrated state completes remaining moves on next boot", () => { + const t = tmpPaths(); + try { + writeFile(join(t.runtimeDir, "nectar.json"), '{"already":"new"}\n'); + writeFile(join(t.legacyDir, "pending-reviews.json"), '["legacy"]\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + const result = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + assert.ok(result.moved.includes("pending-reviews.json")); + assert.equal(existsSync(join(t.runtimeDir, "nectar.json")), true); + assert.equal(existsSync(join(t.runtimeDir, "pending-reviews.json")), true); + assert.equal(existsSync(join(t.legacyDir, "pending-reviews.json")), false); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("c-AC-4 a malformed registry fails loudly (logged, not clobbered) but the migration pass is fail-soft: no throw, refresh skipped, marker written", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "nectar.json"), '{"k":"v"}\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + const malformed = join(t.fleetRoot, "registry.json"); + writeFile(malformed, "{ malformed json "); + + const logged: Array> = []; + let result: ReturnType | undefined; + assert.doesNotThrow(() => { + result = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + log: (line) => logged.push(line), + }); + }); + // Not clobbered (the registerWithDoctor fail-loud-on-write posture stands). + assert.equal(readFileSync(malformed, "utf8"), "{ malformed json "); + // The refresh was skipped, the reason logged loudly with the path. + assert.equal(result?.refreshedRegistry, false); + const warning = logged.find((l) => l["msg"] === "doctor registry refresh skipped; the registry file could not be safely edited (not clobbered)"); + assert.ok(warning !== undefined, "the malformed-registry reason is logged loudly"); + assert.equal(warning?.["registryPath"], malformed); + assert.ok(String(warning?.["err"]).length > 0, "the log carries why the edit was refused"); + // The pass otherwise completed: files moved and the marker written. + assert.ok(result?.moved.includes("nectar.json")); + assert.equal(existsSync(join(t.runtimeDir, MIGRATION_MARKER_FILE_NAME)), true); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("c-AC-4 the install verb keeps the fail-loud write posture: registerWithDoctor still throws DoctorRegistryError on a malformed registry", () => { + const t = tmpPaths(); + try { + mkdirSync(t.fleetRoot, { recursive: true }); + const malformed = join(t.fleetRoot, "registry.json"); + writeFile(malformed, "{ malformed json "); + assert.throws( + () => + registerWithDoctor({ + config: { host: "127.0.0.1", port: 3854, pidFilePath: join(t.runtimeDir, "nectar.pid") }, + registryPath: malformed, + }), + DoctorRegistryError, + ); + assert.equal(readFileSync(malformed, "utf8"), "{ malformed json "); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("c-AC-4 a present-but-malformed doctor registry does not prevent daemon start (boot survives, /health serves)", async () => { + const t = tmpPaths(); + const fakeHome = join(t.root, "home"); + const saved = { + APIARY_HOME: process.env["APIARY_HOME"], + NECTAR_RUNTIME_DIR: process.env["NECTAR_RUNTIME_DIR"], + HOME: process.env["HOME"], + USERPROFILE: process.env["USERPROFILE"], + }; + try { + // Hermetic HOME: the migration's legacy dir and the registry path both + // derive from env, never the real user home. + mkdirSync(fakeHome, { recursive: true }); + process.env["HOME"] = fakeHome; + process.env["USERPROFILE"] = fakeHome; + process.env["APIARY_HOME"] = t.fleetRoot; + delete process.env["NECTAR_RUNTIME_DIR"]; + + // Legacy state exists (triggers the refresh) and the registry is malformed. + writeFile(join(fakeHome, ".honeycomb", "nectar.json"), '{"k":"v"}\n'); + mkdirSync(t.fleetRoot, { recursive: true }); + const malformed = join(t.fleetRoot, "registry.json"); + writeFile(malformed, "{ malformed json "); + + const daemon = assembleDaemon({ port: 0, log: () => {} }); + let port = 0; + try { + port = await daemon.start(); // pre-fix this threw DoctorRegistryError and bricked boot + assert.ok(port > 0, "the daemon bound a port despite the malformed registry"); + assert.equal(readFileSync(malformed, "utf8"), "{ malformed json ", "the malformed registry is never clobbered"); + } finally { + await daemon.shutdown(); + } + } finally { + for (const [name, value] of Object.entries(saved)) { + if (value === undefined) delete process.env[name]; + else process.env[name] = value; + } + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("registry refresh advertises the LEGACY telemetryDbPath when the SQLite move failed and the legacy DB is still live", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "telemetry", "nectar.sqlite"), "sqlite"); + mkdirSync(t.fleetRoot, { recursive: true }); + const result = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + migrateFile: (_sourcePath, targetPath) => { + if (targetPath.endsWith("nectar.sqlite")) throw new Error("simulated copy failure"); + }, + }); + assert.deepEqual(result.failed, ["telemetry/nectar.sqlite"]); + assert.equal(existsSync(join(t.legacyDir, "telemetry", "nectar.sqlite")), true, "the legacy DB is still live"); + + const registry = JSON.parse(readFileSync(join(t.fleetRoot, "registry.json"), "utf8")); + const entry = (registry.daemons as Array<{ name: string; telemetryDbPath: string }>).find((e) => e.name === "nectar"); + assert.equal( + entry?.telemetryDbPath, + join(t.legacyDir, "telemetry", "nectar.sqlite"), + "doctor is pointed at the DB that actually exists, not a not-yet-existing new path", + ); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("registry refresh advertises the NEW telemetryDbPath once the SQLite move succeeded (heals on the retry boot)", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "telemetry", "nectar.sqlite"), "sqlite"); + mkdirSync(t.fleetRoot, { recursive: true }); + const result = runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + assert.deepEqual(result.failed, []); + assert.ok(result.moved.includes("telemetry/nectar.sqlite")); + + const registry = JSON.parse(readFileSync(join(t.fleetRoot, "registry.json"), "utf8")); + const entry = (registry.daemons as Array<{ name: string; telemetryDbPath: string }>).find((e) => e.name === "nectar"); + assert.equal(entry?.telemetryDbPath, join(t.runtimeDir, "telemetry", "nectar.sqlite")); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); + +test("c-AC-4 registry refresh runs after telemetry move", () => { + const t = tmpPaths(); + try { + writeFile(join(t.legacyDir, "telemetry", "nectar.sqlite"), "sqlite"); + mkdirSync(t.fleetRoot, { recursive: true }); + runStateMigration({ + config: baseConfig(t.runtimeDir), + legacyDir: t.legacyDir, + env: { APIARY_HOME: t.fleetRoot }, + homeDir: join(t.root, "home"), + }); + const telemetryMovedAt = statSync(join(t.runtimeDir, "telemetry", "nectar.sqlite")).mtimeMs; + const registryWrittenAt = statSync(join(t.fleetRoot, "registry.json")).mtimeMs; + assert.ok(registryWrittenAt >= telemetryMovedAt); + } finally { + rmSync(t.root, { recursive: true, force: true }); + } +}); diff --git a/test/telemetry-usage.test.ts b/test/telemetry-usage.test.ts index 8f016f1..62d36bf 100644 --- a/test/telemetry-usage.test.ts +++ b/test/telemetry-usage.test.ts @@ -8,7 +8,7 @@ */ import { test } from "node:test"; import assert from "node:assert/strict"; -import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { @@ -269,6 +269,56 @@ test("distinct_id: without install-id, a UUID is minted once, persisted, and sta } }); +test("a-AC-5 install-id read prefers /install-id when both new and legacy files exist", async () => { + const root = tmpStateDir(); + const rec = recorder(); + try { + const fleetRoot = join(root, "apiary"); + const legacyDir = join(root, "legacy"); + mkdirSync(fleetRoot, { recursive: true }); + mkdirSync(legacyDir, { recursive: true }); + writeFileSync(join(fleetRoot, INSTALL_ID_FILE_NAME), "new-root-id\n", "utf8"); + writeFileSync(join(legacyDir, INSTALL_ID_FILE_NAME), "legacy-id\n", "utf8"); + + const outcome = await emitInstalled({ + fetch: rec.fetch, + posthogKey: "phc_test_key", + env: { APIARY_HOME: fleetRoot }, + version: "1.2.3", + apiaryRootDir: fleetRoot, + legacyRuntimeDir: legacyDir, + }); + assert.equal(outcome.sent, true); + assert.equal(rec.calls[0]!.body["distinct_id"], "new-root-id"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("a-AC-5 install-id read falls back to legacy ~/.honeycomb/install-id when fleet-root file is absent", async () => { + const root = tmpStateDir(); + const rec = recorder(); + try { + const fleetRoot = join(root, "apiary"); + const legacyDir = join(root, "legacy"); + mkdirSync(legacyDir, { recursive: true }); + writeFileSync(join(legacyDir, INSTALL_ID_FILE_NAME), "legacy-id\n", "utf8"); + + const outcome = await emitInstalled({ + fetch: rec.fetch, + posthogKey: "phc_test_key", + env: { APIARY_HOME: fleetRoot }, + version: "1.2.3", + apiaryRootDir: fleetRoot, + legacyRuntimeDir: legacyDir, + }); + assert.equal(outcome.sent, true); + assert.equal(rec.calls[0]!.body["distinct_id"], "legacy-id"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + // ── Fail-soft: never throws, never alters exit codes ───────────────────── test("fail-soft: a rejecting fetch resolves to send_failed, never throws", async () => { diff --git a/test/telemetry/db.test.ts b/test/telemetry/db.test.ts index 341136b..51da210 100644 --- a/test/telemetry/db.test.ts +++ b/test/telemetry/db.test.ts @@ -18,13 +18,13 @@ function tmpDir() { } test("telemetryDbPathForRuntimeDir nests the db under telemetry/nectar.sqlite", () => { - const p = telemetryDbPathForRuntimeDir("/home/op/.honeycomb"); - assert.equal(p, join("/home/op/.honeycomb", TELEMETRY_DIR_NAME, TELEMETRY_DB_FILE_NAME)); + const p = telemetryDbPathForRuntimeDir("/home/op/.apiary/nectar"); + assert.equal(p, join("/home/op/.apiary/nectar", TELEMETRY_DIR_NAME, TELEMETRY_DB_FILE_NAME)); }); -test("defaultTelemetryDbPath nests under /.honeycomb/telemetry/nectar.sqlite", () => { +test("defaultTelemetryDbPath nests under /.apiary/nectar/telemetry/nectar.sqlite", () => { const p = defaultTelemetryDbPath("/home/op"); - assert.equal(p, join("/home/op", ".honeycomb", "telemetry", "nectar.sqlite")); + assert.equal(p, join("/home/op", ".apiary", "nectar", "telemetry", "nectar.sqlite")); }); test("openTelemetryDb creates the directory and the file, and is idempotent to re-open", () => {