feat(carve S7): the orchestration compute plane — reconciler, release, rollup

[isadeks]

Slice 7 of the staged carve. Stacked on #657 (carve/s6-decompose-iteration) — review that first; this PR's diff is only its own commit.

What this adds

The engine that actually runs a sub-issue graph. It watches child tasks reach a terminal state, releases whatever they unblocked, keeps the parent's status panel current, and re-stacks dependents when a branch moves.

• orchestration-discovery — the composer that turns a trigger into a seeded run: read the graph, validate it, append the integration node if it fans out, persist it, and hand back the root set to start.
• orchestration-reconcile — pure gating. Given a child that just went terminal, decide which dependents are now releasable, which must be skipped, and when the parent itself is done. Two recovery shapes are covered too: comment-fixing a failed child un-fails it and re-releases the dependents that were skipped behind it, and re-triggering a finished epic retries only its failed/skipped children.
• orchestration-release — create a child's task, behind a claim so two concurrent releasers can't both launch it. A guardrail rejection or an un-onboarded repo is terminal with a reason the user can act on; a 5xx or a duplicate replay rolls back to ready so the sweep retries.
• orchestration-rollup — the parent's single maturing panel: one comment that matures in place rather than a stream of updates. A failed row carries an indented sub-line saying what failed and where to read it.
• orchestration-restack — when a child's branch changes, work out which dependents need their base moved.
• orchestration-parent-comment — route a comment on the epic to the sub-issue it's actually about, and say so plainly when it looks like new work rather than a change to existing work.
• orchestration-reconciler (handler + construct) — the TaskTable-stream consumer that drives all of the above. A DLQ for poison records, a FilterCriteria so only terminal-status writes invoke it (RUNNING/heartbeat churn is the bulk of TaskTable writes and would otherwise wake it constantly), and partial-batch reporting so one bad record can't re-drive its healthy siblings.
• reconcile-stranded-orchestrations (handler + construct) — a scheduled backstop for a run whose stream event was lost.

⚠️ Unlike S5 and S6, this slice is NOT fully dormant

TaskTable gains a LinearIssueIndex GSI and has its stream enabled (NEW_IMAGE). TaskTable is instantiated in the stack, so these two changes do land on deploy.

Both are required by this slice and can't be deferred: the reconciler consumes that stream, and it queries that index to total an issue's iteration cost. Notes for review:

• Enabling a stream and adding a GSI are both in-place CloudFormation updates — no table replacement.
• The GSI projection is deliberately narrow (pr_url, pr_number, status, repo, user_id, channel_metadata). A GSI's projection cannot be changed in place afterwards, which is why the cost query does a per-task GetItem for cost_usd rather than widening it.
• The stream is on TaskTable rather than TaskEventsTable because the latter is already at its 2-consumer limit.
• The reconciler constructs themselves are not instantiated in any stack yet — that wiring is the next slice — so nothing consumes the stream on deploy.

Verification

• tsc --noEmit clean
• CDK: 3491 tests / 173 suites passing (up from 3213/164 on the base)
• eslint clean

🤖 Generated with Claude Code

---

Tracking

Slice S7 of the linear-vercel → main carve. Tracking issue: #668 (slice table, why it is sliced rather than merged, and review guidance).

Lands part of #247 (parent/sub-issue orchestration). Deliberately not Closes — no single slice closes it; it closes when S8 activates the arc. The auto-decomposition issue (#299) is no longer part of this carve — that feature stays on the development branch as experimental.

Stacked on #657 — review that first; this PR's diff is against its branch.

[isadeks]

Automated review — carve S7 (#659)

Reviewed as its own diff only (carve/s6-decompose-iteration..carve/s7-orchestration-plane, 21 files, +10274/−4). This PR does not claim dormancy — it enables a DynamoDB stream and adds a GSI on the live TaskTable — so the CFN-safety questions got priority. Governance is noted once for the whole stack, not repeated here.

Slice standalone-ness: PASS

tsc --noEmit clean at this tip on its own base (compiler run, not inferred). Only 4 deletions in the whole diff. Every new source file ships with its test in this same slice — no source/test split at this boundary.

---

CFN safety — the questions worth asking before this merges

Stream enablement: in-place, no replacement. CONFIRMED.
task-table.ts adds stream: dynamodb.StreamViewType.NEW_IMAGE. StreamSpecification is an in-place update on AWS::DynamoDB::Table, and I checked that nothing else in the same diff touches a replacement-forcing property — partitionKey (task_id), sortKey, billingMode (PAY_PER_REQUEST), and tableName (still props.tableName passthrough) are all unchanged. The comment in the code makes the same claim and it holds up.

GSI addition: one per deploy. CONFIRMED.
DynamoDB permits one GSI create per update, and this diff adds exactly one (LinearIssueIndex), taking TaskTable from 4 GSIs to 5:

ref	GSIs	names
main / s6	4	UserStatus, Status, Idempotency, JiraIssue
s7 (#659)	5	+ LinearIssueIndex
s8	5	unchanged

Since JiraIssueIndex already exists on main, this is a clean single-index addition, not two-at-once.

GSI projection: correct for its purpose — but four call sites already pay for it.
Projection is INCLUDE with ['pr_url','pr_number','status','repo','user_id','channel_metadata']. I traced every consumer and every attribute it reads:

• linear-task-by-issue.ts resolveTaskByLinearIssue reads task_id, user_id, repo, pr_url, pr_number, status — all projected ✅
• github-webhook-processor.ts reads task_id, channel_metadata.iteration_reply_comment_id — projected ✅

So the routing lookups the index was designed for are fully covered, and the projection is right for them. But four call sites need attributes the index does not project, and each pays a per-item base-table GetItem:

• orchestration-reconciler.ts:1266 — cost_usd
• fanout-task-events.ts sumIterationCostForIssue — cost_usd
• clarify-resume.ts:45 — code_changed, answer_text, task_description, workflow pin
• linear-webhook-processor.ts:3747 — the clarify fields (reads the full base row)

Each is deliberate and documented in-code. I am raising it because a GSI projection cannot be changed in place — DynamoDB rejects it, as the code comment itself records from experience. So this list is the permanent cost of the current shape. Four known consumers already working around it, two of them for the same field (cost_usd), is the moment to ask whether cost_usd belongs in the projection. Adding it now is free; adding it later means creating a second index. Worth a deliberate decision rather than inheriting it.

---

On the previously-flagged sumIterationCostForIssue concern — largely REFUTED

The standing concern was an unpaginated Query on LinearIssueIndex plus serial per-task GetItem in a stream-consumer Lambda. I traced it in both copies and the batch-multiplication part does not hold:

• The Query is unpaginated (no LastEvaluatedKey loop) and the GetItem loop is serial. Both confirmed.
• But it does not run per stream record. Three gates precede it in replyToStandaloneTrigger: it requires a trigger comment, it excludes orchestration iterations, and it takes a one-shot claim — UpdateExpression: 'SET ack_replied_at = :now' with ConditionExpression: 'attribute_not_exists(ack_replied_at)'. So a redelivered batch (fan-out batchSize: 100, reconciler batchSize: 10) is claimed away and the sum runs once per terminal iteration task.
• Realistic N is the number of iterations a human drives on one issue — single digits. At that size, serial GetItem is a few milliseconds.
• Both consumers are reportBatchItemFailures: true, so a poison record doesn't block the shard.

Residual risk, small but real: at the 1 MB Query page limit the function silently undercounts rather than erroring — total is summed from a truncated Items list and returned as if complete. That needs roughly thousands of tasks on one issue to trigger, so I would not block on it. If you want the cheap fix, it is a LastEvaluatedKey loop or a Limit plus an explicit "showing first N" note, so the number shown to the user is never quietly wrong.

Duplication worth noting: sumIterationCostForIssue exists twice with the same shape — fanout-task-events.ts and orchestration-reconciler.ts:1252. Two copies of a cost-accounting function will drift.

---

Other findings

MINOR — renderStatusBlock, rollupKindFromChildren, renderRollupComment, postRollup, renderEpicPanel, buildPanelRows and truncateQuote are exported from orchestration-rollup.ts but have zero callers in src/ anywhere in the stack.
Verified at the s8 tip: git grep -l <fn> -- 'cdk/src/**' excluding the defining file returns nothing for all seven. Only upsertEpicPanel and cascadeNodeLabel are actually used. The file's own comment explains why — the maturing panel "supersedes the separate renderStatusBlock + renderRollupComment" — so this is superseded code that shipped alongside its replacement. It is invisible to the dead-code ratchet because each function has a test file referencing it, and knip-baseline.json is unchanged at 78 across the whole stack. ~600 lines of dead rendering logic is a lot to publish; consider deleting the superseded half.

I also rendered the two dead functions to check whether they were merely unused or actually wrong, and they are wrong in ways that would matter if anything called them:

renderStatusBlock([succeeded, failed, blocked, released, skipped])
  → "🔄 **ABCA orchestration** · 3/5 complete"

Three of five "complete" with one success — terminal() counts failed and skipped toward "complete".

rollupKindFromChildren([succeeded, released])  → "complete"
rollupKindFromChildren([])                      → "complete"

A still-released (running) child yields complete, and an empty set yields complete. Another reason to delete rather than keep.

MINOR — a raw internal UUID is shown to users on the fallback path. In renderStatusBlock, when a child has neither display_id nor title the label falls back to sub_issue_id, rendering - ✅ issue-uuid-1 — succeeded. Currently unreachable (dead function), but the same display_id ?? sub_issue_id fallback pattern is worth checking in renderEpicPanel before activation.

IAM

Least-privilege on the new roles looks sound: grantReadData where read-only suffices, grantReadWriteData only on the three tables the reconciler genuinely writes, a DLQ with enforceSSL and 14-day retention for poison records, and cdk-nag suppressions that name the CDK-generated index/* wildcards specifically rather than blanket-suppressing. No wildcard actions. The one over-broad grant I found is in #662 (traceArtifactsBucket.grantRead on the whole bucket) and is reported there.

---

Reviewed with Claude Code. Verification: tsc --noEmit on this slice's own base; GSI count and stream declaration compared across six refs; every LinearIssueIndex consumer's attribute reads traced against the projection list; the claim gate read as an actual ConditionExpression; the dead rollup functions executed to check their output.

[isadeks]

Addendum to the review above — two further findings

A second pass surfaced two things the first review missed.

MINOR (user-copy + public-fitness) — a user-facing retry instruction names the wrong label.

orchestration-rollup.ts:318:

(No reply? Removing and re-applying the `abca` label also retries.)

The trigger label is per-project configurable (label_filter on the project-mapping row) and its default is bgagent — DEFAULT_LABEL_FILTER = 'bgagent' in orchestration-decomposition-mode.ts:44, linear-webhook-processor.ts:35 and jira-webhook-processor.ts:58, and the docs say the same. abca is this project's own label, not the shipped default.

So a user who follows this instruction removes and re-applies a label the webhook does not filter on, and nothing happens. It should interpolate the resolved label_filter for that project, or fall back to DEFAULT_LABEL_FILTER rather than a hardcoded string. This is both a correctness bug in user-facing copy and an instance of the use-case-specific-content class.

NIT — taskTableForWrites is a prop that silently does nothing.

OrchestrationReconcilerProps.taskTableForWrites (orchestration-reconciler.ts:46) is declared and documented ("TaskTable (for createTaskCore writes when releasing children)"), but the constructor never reads it — the write grant actually comes from props.taskTable.grantReadWriteData(this.fn). It is also never passed by the stack at #662.

Worth removing rather than leaving: the only reason a caller would reach for this prop is to point writes at a different table, and doing so would silently have no effect while appearing to work.

NIT — third instance of the "cited artifact lands later" pattern.

reconcile-stranded-orchestrations.ts:39 and orchestration-release.ts both refer the reader to docs/research/orchestration-reconciler-correctness.md for the failure-mode analysis justifying the sweep. That file does not exist at main, s1–s6, or this slice — it is added in #662. So this slice's two most safety-critical modules cite a nonexistent document for their correctness rationale.

This is the same shape as the ADR-016 citations in #654 and the model/IAM split in #654 → #662. See the stack-level note in the summary: the recurring boundary error in this stack is code landing in an earlier slice than the artifact that authorises it.

[isadeks]

Review findings addressed

Thank you for the CFN-safety pass — checking stream enablement and GSI count against six refs is exactly the verification this slice needed, and I'm glad the in-place claims held up under it.

MINOR — user-facing retry copy named the wrong label. Fixed. This was the most valuable finding in the review: the hint told users to re-apply a hardcoded label while the shipped default is a different string and the label is per-project configurable, so a user following the instruction got silence. renderEpicPanel and upsertEpicPanel now take the resolved label and fall back to the platform default. I rendered the panel to read the output rather than trusting the source.

Note the existing test asserted the hardcoded string, which is why this shipped — so I fixed the assertion too and added two cases (a custom label, and the default fallback). All three fail if the hardcoded string comes back.

MINOR — taskTableForWrites is a prop that silently does nothing. Confirmed, removed. Your reasoning for removing rather than leaving it is right: the only reason to reach for it is to point writes at a different table, and that would appear to work while having no effect.

GSI projection — cost_usd. Genuinely useful framing: adding it now is free, later means a second index. I have not added it, and want to be explicit that this is a judgement call rather than an oversight. Four consumers pay a GetItem, two for that one field — but widening the projection makes every task write carry cost_usd into the index, and the field is written late and often on iteration-heavy tasks. The GetItem cost is bounded per your own analysis below; the write amplification is not as easy to bound. If you disagree I will take the change — you have looked at the consumers more closely than the original author did.

sumIterationCostForIssue — thank you for refuting the part that did not hold. The claim as I had it (batch multiplication in a stream consumer) was wrong, and reading the claim gate as an actual ConditionExpression is the check I should have made. The residual you identified — a silent undercount at the 1 MB page boundary rather than an error — is the real defect, correctly sized as non-blocking. Not fixed here; tracked, with your suggested fix (page loop, or an explicit "first N" note so the number shown is never quietly wrong).

Duplicated sumIterationCostForIssue. Correct, two copies with the same shape. Not de-duplicated in this slice because the copies live either side of a slice boundary; folding them together is a follow-up, and the page-limit fix should land in one place once they are merged.

MINOR — ~600 lines of dead rendering logic. Confirmed: seven exports with no src/ callers, invisible to the dead-code ratchet because each has a test. Not deleted here — that is a clean subtractive change, and I would rather it be its own reviewable commit than buried in a slice this size. Your rendering of the two dead functions is the part I want to flag as valuable: rollupKindFromChildren([]) returning complete, and terminal() counting failures toward "3/5 complete", would both be real bugs if anything called them. That turns "delete this" from tidiness into correctness, and I have recorded it that way.

MINOR — raw UUID on the fallback path. Checked renderEpicPanel's live path as you suggested; it uses the same display_id ?? sub_issue_id shape, so the exposure is real there too if a row lacks both. Left as-is for now (a row with neither is a data defect in its own right) but noted.

NIT — third instance of "cited artifact lands later". Your stack-level observation is the most useful thing in this review. Two of the three are addressed (the descriptor/YAML split, the model/grant split); the doc citations I have left, with reasoning on #654. The pattern is real and worth stating as a slicing rule: code must not land in an earlier slice than the artifact that authorises it.

[isadeks]

Operator note: this slice adds one GSI, but a STALE deployment may need two — DynamoDB allows one per update

Found while deploying this arc to a second environment. Correcting something I asserted earlier in this PR.

What I verified, and why it was the wrong baseline. I checked that TaskTable goes from 4 GSIs to 5 — adding only LinearIssueIndex — and that JiraIssueIndex already existed on main. Both true: JiraIssueIndex came in with #640, which is an ancestor of this carve's base, so it is not this arc's index. This slice adds exactly one GSI relative to main.

But DynamoDB does not constrain the delta against main. It constrains the delta against what is actually deployed:

Cannot perform more than one GSI creation or deletion in a single update

An environment whose stack predates #640 has three GSIs live (StatusIndex, IdempotencyIndex, UserStatusIndex). Deploying this branch there asks for two new indexes at once — JiraIssueIndex and LinearIssueIndex — which DynamoDB refuses. TaskTable goes UPDATE_FAILED and the whole stack rolls back. (The rollback is clean: the table is not replaced, and the newly-created orchestration resources are removed.)

The dev account did not surface this only because its stack was already current with main before the arc went on. That is exactly the kind of thing that hides on the machine where the work was done.

Operator guidance for merging/deploying this:

• Confirm the target stack is current with main before deploying this slice. If it is not, deploy main first (which adds JiraIssueIndex alone), then this branch (which adds LinearIssueIndex alone).
• This is per-deployment state, not a code change — nothing in the template needs fixing, and splitting the index across slices would not help, since the constraint is about the deployed baseline rather than the source.

Also worth recording, since it bears on how anyone verifies this: on that environment two cdk deploy runs printed exit code 0 while failing at Docker image build (a NodeSource CDN 403 that then poisoned the layer cache — pruning the build cache cleared it). Same trap hit on the dev account, where two runs exited 0 while failing at ECR asset publish. Check the CloudFormation stack status and LastUpdatedTime, not the CLI's exit code.

[isadeks]

Second-round review: my retry-hint fix was cosmetic — now actually wired

The reviewer checked whether any production caller passes the labelFilter I added. None did, and none could: all three upsertEpicPanel call sites omit it, and the reconciler has no project-mapping access — no mapping table in its props, and no mapping env var. So every user still saw the platform default and the hint was still wrong for any project that renamed its trigger label. The original finding was still open in production; my fix only moved the parameter into place.

Fixed properly. The label is now captured where the mapping is in hand — at seed time — and persisted on the meta row's release context; the reconciler reads it back and passes it to the panel. Rows seeded before the field existed simply have none and fall back to the platform default, as before.

Only the settled-panel call site is wired, deliberately: the retry hint renders only when the epic is terminal and something failed, and the other two sites render an in-progress panel that has no hint. I verified that rather than assuming it.

Guarded in both directions, and the read half is the one that mattered: dropping the write fails the seed test, but dropping the read hydration originally left the entire 3500-test suite green. Both are now pinned.

On taskTableForWrites — fair criticism that an unexplained API removal was bundled into a copy fix. It is genuinely dead (no caller on this branch, the next, or the source branch) and removing it was the right call, but it belonged in its own commit with the reasoning stated. Noted.

On the labelFilter markdown escaping — accepted as low severity and not changed: the value comes from a project mapping an admin sets, not from user input. Worth a backtick strip for consistency; tracked rather than done here.

Verified and not changed: the panelRow rename is clean and load-bearing (mutating panelLabel to ignore display_id fails 5 of the renamed tests), and the backward dependency on the decomposition module resolves at runtime, not just under tsc.

[isadeks]

Upgrade-ordering note added, from a second-environment deploy — and a correction

Deployed this arc to a different account (one whose stack was last updated at pre-#643 main) and hit the GSI limit I had reasoned about earlier from the wrong baseline:

Cannot perform more than one GSI creation or deletion in a single update

I had verified "exactly one new GSI" against main, which is true — this slice adds only LinearIssueIndex, and JiraIssueIndex came from #640, an ancestor of the carve base. But DynamoDB constrains the delta against what is deployed, not against main. A stack predating #640 has three GSIs live, so this branch asks for two at once and CloudFormation rolls the whole stack back.

Operator guidance: confirm the target stack is current with main before deploying this slice. If it is not, deploy main first, wait for JiraIssueIndex to reach ACTIVE (~10 min), then deploy this branch. Verified working in that order — UPDATE_COMPLETE, table TableId unchanged so no replacement, and the resulting GSI set is the four pre-existing plus LinearIssueIndex.

Nothing in the template needs changing, and splitting the index across slices would not help — the constraint is about the deployed baseline.

A correction to something I wrote earlier in this thread. I claimed a stray artifact URI "would fail closed anyway" because the reconciler's grant is scoped to artifacts/*. That is wrong, and I confirmed it against the live policy: S3 does not normalize keys, so artifacts/../traces/u/x is a literal key that an artifacts/* resource matches by string prefix — iam simulate-principal-policy returns allowed for exactly that shape. So the handler-side check was the only thing standing between a tampered task record and another user's trajectory, and a bare startsWith did not cover traversal. Now fixed to reject traversal segments (raw and percent-encoded), with a test — deleting the check previously left all 64 reconciler tests green.

[isadeks]

Scope change: auto-decomposition is no longer part of this carve

Heads-up before you re-read this — every slice was force-pushed, so a review in progress is against a stale diff. Sorry for the churn; the reason is a scope decision, not a fix.

Auto-decomposition is staying on the development branch as experimental and is not landing on main. It presupposes one way of working: an issue split into a sub-issue graph, with a human approval gate on a proposed plan. main should not presuppose that. Everything else in this arc is workstyle-neutral — running a graph a human already authored, iterating on a PR from a comment, heartbeats, clarify-resume, the Channel abstraction, the compute plane — and those all stay.

What moved

~7,900 lines removed. 23 dedicated files, the agent workflow and its prompt, the admission-table entry, the CLI flags, and about 314 lines embedded in files that survive.

slice	effect
#647 S1	unchanged
#653 S2	comments now describe artifact workflows by their contract, not by one workflow
#654 S3	workflow YAML, prompt, and admission entry removed
#655 S4	--decompose-allowed / --max-sub-issues / --max-parent-budget-usd removed from linear onboard-project
#656 S5	note-sweep renamed for what it does
#657 S6	8538 → ~2000 lines. Now purely the iteration slice; retitled
#659 S7	graph CREATION removed from the reconciler; graph EXECUTION kept
#662 S8	the plan-proposal surface removed; webhook 4502 → 3156 lines

Three things worth knowing as a reviewer

Some of the review fixes from the last round are gone with the feature. The verdict-marker fix (B1 on #662) and the retry-hint trigger_label fix (#659) were both decomposition-only. They were real fixes and they live on the development branch — they are not being dropped as wrong.

Two things looked decomposition-only and were not. DEFAULT_LABEL_FILTER and hasHelpLabel lived in a decompose-named module but are plain label plumbing, so they moved to trigger-label.ts rather than being deleted. And the note sweep is prefix-based and general, so it was renamed, not removed. Both would have been silent breakage.

Removing the feature exposed dead IAM. The reconciler held an s3:GetObject on the artifacts prefix plus put/delete on the attachments bucket, all for plan seeding. They had no caller left. Removed — a net least-privilege gain, and there is now a test asserting the reconciler has no object grant at all, which is the stronger claim since S3 does not normalize keys.

Verification

• All 8 slices CI-green, chain integrity confirmed (each slice's base is its parent's tip)
• 3436 CDK / 681 CLI / 1436 agent tests at the tip
• Zero decompose references in every slice individually, not just at the tip — an earlier pass looked clean at the tip while intermediate slices still carried them, which would have landed decompose text on main temporarily during the serial merge
• Deployed to dev and live-verified by artifact: the deployed webhook bundle contains no plan surface; coding/decompose-v1 is now rejected at admission (Unknown workflow_ref, 400); a normal coding task still runs end to end (PR opened with the requested file, exact contents); coding/pr-review-v1 still admits; the reconciler's IAM shows no S3 statements