feat(skill): orchestrate dependency-aware story waves - #125
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdded the ChangesStory Wave Orchestration
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Coordinator
participant ImplementerThread
participant ReviewerThread
participant Repository
Coordinator->>Repository: inspect repository and tracker state
Coordinator->>ImplementerThread: dispatch eligible story in isolated worktree
ImplementerThread->>Repository: implement and provide candidate evidence
Coordinator->>ReviewerThread: request bounded review
ReviewerThread->>Coordinator: return review result and follow-up requirements
Coordinator->>Repository: verify candidate and deliver PR or integration
Coordinator->>Coordinator: recompute readiness and completion state
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 134f1ba502
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/orchestrate-story-waves/evals/evals.json:
- Around line 19-25: Update the integration-mode eval assertions to explicitly
verify readiness is recomputed after each terminal integration and before
dispatching dependent stories, matching the readiness-refresh requirement in the
wave protocol. Keep the existing assertions unchanged and add only the missing
readiness-recomputation contract.
- Around line 30-37: Update the evaluation prompt to state that the story is
admitted while provider evidence is current, then becomes blocked when that
evidence is revoked or turns stale; preserve the existing expectations about
holding only its descendants and continuing independent eligible stories. Apply
this change to the prompt text in the relevant eval entry, keeping its
assertions aligned with the wave-protocol eligibility contract.
- Around line 9-13: Add an assertion to the evals.json assertions list that
verifies dispatch never exceeds the user-requested three-story limit, ensuring
no fourth or later story is dispatched. Keep the existing completion-count
assertion and other dispatch/threading assertions unchanged.
In @.agents/skills/orchestrate-story-waves/references/thread-dispatch.md:
- Around line 26-28: Clarify model-override precedence for implementer and
reviewer roles in both SKILL.md and thread-dispatch.md. State whether
user-specified overrides apply to implementers, reviewers, or both, then update
the reviewer model requirement and corresponding dispatch guidance so both
documents enforce the same rule.
In @.agents/skills/orchestrate-story-waves/references/wave-protocol.md:
- Around line 19-25: The wave protocol must advance after every terminal story
outcome, including blocked or failed outcomes. Update the
initialization/terminal-merge procedure to refresh state and the ledger,
recompute eligibility, and launch all eligible stories within capacity; pause
only descendants of the blocked story.
- Around line 47-49: Update the integration protocol after PASS to explicitly
require fast-forward or no-fast-forward integration, then verify the reviewed
commit remains an ancestor of the resulting integration commit with git
merge-base --is-ancestor. If conflict resolution creates a new candidate commit,
rerun all required checks and obtain independent reviewer validation for that
exact commit.
In @.agents/skills/orchestrate-story-waves/SKILL.md:
- Line 89: Update the validation step in the story-waves skill instructions to
use a validator configured to accept “orchestrate-story-waves” rather than the
hard-coded “orchestrate-phase-delivery” value, and record the exact command used
to validate the skill and its eval fixtures.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 8ea79578-b17e-4580-bf85-0e3f9d7b88f1
📒 Files selected for processing (5)
.agents/skills/orchestrate-story-waves/SKILL.md.agents/skills/orchestrate-story-waves/evals/evals.json.agents/skills/orchestrate-story-waves/evals/trigger_queries.json.agents/skills/orchestrate-story-waves/references/thread-dispatch.md.agents/skills/orchestrate-story-waves/references/wave-protocol.md
| "assertions": [ | ||
| "Only tracker-eligible stories are dispatched", | ||
| "Every story uses separate implementer and read-only reviewer threads", | ||
| "The next wave starts only after verified merge", | ||
| "Exactly three verified story completions are counted" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Enforce the three-story dispatch boundary.
The prompt requests three stories, but the assertions only verify that three completions are counted. An implementation can dispatch a fourth story and still satisfy the current assertions. Add an assertion that no story beyond the requested boundary is dispatched.
Reference: .agents/skills/orchestrate-story-waves/SKILL.md requires dispatch within the user’s story-count limit.
Suggested assertion
"assertions": [
"Only tracker-eligible stories are dispatched",
"Every story uses separate implementer and read-only reviewer threads",
"The next wave starts only after verified merge",
- "Exactly three verified story completions are counted"
+ "Exactly three verified story completions are counted",
+ "No story beyond the requested three-story boundary is dispatched"
]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "assertions": [ | |
| "Only tracker-eligible stories are dispatched", | |
| "Every story uses separate implementer and read-only reviewer threads", | |
| "The next wave starts only after verified merge", | |
| "Exactly three verified story completions are counted" | |
| "assertions": [ | |
| "Only tracker-eligible stories are dispatched", | |
| "Every story uses separate implementer and read-only reviewer threads", | |
| "The next wave starts only after verified merge", | |
| "Exactly three verified story completions are counted", | |
| "No story beyond the requested three-story boundary is dispatched" |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/evals/evals.json around lines 9 - 13,
Add an assertion to the evals.json assertions list that verifies dispatch never
exceeds the user-requested three-story limit, ensuring no fourth or later story
is dispatched. Keep the existing completion-count assertion and other
dispatch/threading assertions unchanged.
| "expected_output": "Integration mode preserves reviewed story commits, recomputes readiness after integration, and creates one final phase PR with its lifecycle.", | ||
| "files": [], | ||
| "assertions": [ | ||
| "No per-story PR is created", | ||
| "Every candidate receives independent read-only review", | ||
| "Story commits remain ancestors of the integration result", | ||
| "Only one final phase PR is created" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Assert readiness recomputation before dependent dispatch.
The expected output requires readiness recomputation, but the assertions do not check it. An implementation can integrate commits and create one final PR while dispatching dependent stories from stale readiness. Add an explicit assertion for this contract.
Reference: .agents/skills/orchestrate-story-waves/references/wave-protocol.md requires readiness refresh after each terminal integration.
Suggested assertion
"assertions": [
"No per-story PR is created",
"Every candidate receives independent read-only review",
"Story commits remain ancestors of the integration result",
- "Only one final phase PR is created"
+ "Only one final phase PR is created",
+ "Readiness is recomputed from verified integration evidence before the next dependent story is dispatched"
]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "expected_output": "Integration mode preserves reviewed story commits, recomputes readiness after integration, and creates one final phase PR with its lifecycle.", | |
| "files": [], | |
| "assertions": [ | |
| "No per-story PR is created", | |
| "Every candidate receives independent read-only review", | |
| "Story commits remain ancestors of the integration result", | |
| "Only one final phase PR is created" | |
| "expected_output": "Integration mode preserves reviewed story commits, recomputes readiness after integration, and creates one final phase PR with its lifecycle.", | |
| "files": [], | |
| "assertions": [ | |
| "No per-story PR is created", | |
| "Every candidate receives independent read-only review", | |
| "Story commits remain ancestors of the integration result", | |
| "Only one final phase PR is created", | |
| "Readiness is recomputed from verified integration evidence before the next dependent story is dispatched" |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/evals/evals.json around lines 19 -
25, Update the integration-mode eval assertions to explicitly verify readiness
is recomputed after each terminal integration and before dispatching dependent
stories, matching the readiness-refresh requirement in the wave protocol. Keep
the existing assertions unchanged and add only the missing
readiness-recomputation contract.
| "prompt": "Run six stories autonomously. One ready story becomes blocked by missing provider evidence, while another independent lane remains ready.", | ||
| "expected_output": "Only the blocked story and descendants stop; independent eligible stories continue until the count or governed stop condition.", | ||
| "files": [], | ||
| "assertions": [ | ||
| "The skill does not invent provider evidence", | ||
| "Only descendants of the blocked story are held", | ||
| "Independent eligible stories continue", | ||
| "The final report distinguishes completed count from blocked attempts" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Make the provider-evidence transition explicit.
The prompt says that a ready story becomes blocked because provider evidence is missing. Under the readiness contract, missing provider evidence means that the story is not eligible at the initial check. State that the evidence becomes stale or is revoked after admission. Otherwise, this evaluation contradicts the eligibility rules.
Reference: .agents/skills/orchestrate-story-waves/references/wave-protocol.md defines eligibility using current provider evidence.
Suggested prompt change
- "prompt": "Run six stories autonomously. One ready story becomes blocked by missing provider evidence, while another independent lane remains ready.",
+ "prompt": "Run six stories autonomously. One initially eligible story is later found to have stale provider evidence during state reconciliation, while another independent lane remains eligible.",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "prompt": "Run six stories autonomously. One ready story becomes blocked by missing provider evidence, while another independent lane remains ready.", | |
| "expected_output": "Only the blocked story and descendants stop; independent eligible stories continue until the count or governed stop condition.", | |
| "files": [], | |
| "assertions": [ | |
| "The skill does not invent provider evidence", | |
| "Only descendants of the blocked story are held", | |
| "Independent eligible stories continue", | |
| "The final report distinguishes completed count from blocked attempts" | |
| "prompt": "Run six stories autonomously. One initially eligible story is later found to have stale provider evidence during state reconciliation, while another independent lane remains eligible.", | |
| "expected_output": "Only the blocked story and descendants stop; independent eligible stories continue until the count or governed stop condition.", | |
| "files": [], | |
| "assertions": [ | |
| "The skill does not invent provider evidence", | |
| "Only descendants of the blocked story are held", | |
| "Independent eligible stories continue", | |
| "The final report distinguishes completed count from blocked attempts" |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/evals/evals.json around lines 30 -
37, Update the evaluation prompt to state that the story is admitted while
provider evidence is current, then becomes blocked when that evidence is revoked
or turns stale; preserve the existing expectations about holding only its
descendants and continuing independent eligible stories. Apply this change to
the prompt text in the relevant eval entry, keeping its assertions aligned with
the wave-protocol eligibility contract.
| The implementation thread creates one reviewer thread with model `gpt-5.6-terra` and reasoning | ||
| `high`. The reviewer prompt must bind: | ||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- SKILL.md relevant section ---'
sed -n '30,52p' .agents/skills/orchestrate-story-waves/SKILL.md
printf '%s\n' '--- thread-dispatch.md relevant section ---'
sed -n '1,70p' .agents/skills/orchestrate-story-waves/references/thread-dispatch.md
printf '%s\n' '--- model and override references ---'
rg -n -i 'model|override|reviewer|implementer|gpt-5\.6-terra' \
.agents/skills/orchestrate-story-wavesRepository: agentic-workflow-kit/jig
Length of output: 10224
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- SKILL.md policy and dispatch context ---'
sed -n '1,105p' .agents/skills/orchestrate-story-waves/SKILL.md
printf '%s\n' '--- related policy wording across the repository ---'
rg -n -i -C 2 \
'user-specified model|model override|model overrides|override.*model|model.*override|fixed role defaults|gpt-5\.6-(luna|terra)|planned_model|actual_model' \
.agents README.md 2>/dev/null || trueRepository: agentic-workflow-kit/jig
Length of output: 11686
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
skill = Path(".agents/skills/orchestrate-story-waves/SKILL.md").read_text()
dispatch = Path(".agents/skills/orchestrate-story-waves/references/thread-dispatch.md").read_text()
checks = {
"SKILL.md declares a reviewer default": "Reviewer:" in skill and "gpt-5.6-terra" in skill,
"SKILL.md declares a generic model override": "User-specified model" in skill,
"SKILL.md scopes the override to a role": any(
phrase in skill.lower()
for phrase in ("implementer override", "reviewer override", "per role", "for each role")
),
"thread-dispatch.md requires Terra for reviewers": (
"## Reviewer thread" in dispatch
and "with model `gpt-5.6-terra`" in dispatch
),
"thread-dispatch.md defines reviewer override behavior": any(
phrase in dispatch.lower()
for phrase in ("override", "user-specified model", "configured model", "requested model")
),
}
for name, result in checks.items():
print(f"{name}: {'yes' if result else 'no'}")
PYRepository: agentic-workflow-kit/jig
Length of output: 406
Define model-override precedence per role.
SKILL.md permits user-specified model overrides, but thread-dispatch.md requires gpt-5.6-terra for every reviewer. State whether overrides apply to implementers, reviewers, or both, and apply the same rule in both documents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/references/thread-dispatch.md around
lines 26 - 28, Clarify model-override precedence for implementer and reviewer
roles in both SKILL.md and thread-dispatch.md. State whether user-specified
overrides apply to implementers, reviewers, or both, then update the reviewer
model requirement and corresponding dispatch guidance so both documents enforce
the same rule.
| At initialization and after each terminal merge/integration: | ||
|
|
||
| 1. Refresh the target and hosted state. | ||
| 2. Validate the tracker with its own consumer gate. | ||
| 3. Reconcile the external ledger against registered worktrees and exact commits. | ||
| 4. Mark completed, active, recoverable-blocked, owner-decision-blocked, and eligible stories. | ||
| 5. Launch all eligible stories within pair capacity and the remaining count. |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
Advance waves after every terminal story outcome.
The current trigger excludes blocked or failed stories. A blocked story can release pair capacity while an independent eligible story remains unlaunched. This conflicts with .agents/skills/orchestrate-story-waves/SKILL.md, Lines 67-69, and the blocked-descendant-independent-lane evaluation in .agents/skills/orchestrate-story-waves/evals/evals.json, Lines 26-35.
Refresh the ledger and eligible set after every terminal outcome. Keep only the blocked story's descendants paused.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/references/wave-protocol.md around
lines 19 - 25, The wave protocol must advance after every terminal story
outcome, including blocked or failed outcomes. Update the
initialization/terminal-merge procedure to refresh state and the ledger,
recompute eligibility, and launch all eligible stories within capacity; pause
only descendants of the blocked story.
| 1. Each story gets its own registered worktree and exact-candidate review. | ||
| 2. After `PASS`, integrate the reviewed commit into the integration branch while preserving it as | ||
| an ancestor. Record the resulting integration commit and keep the story worktree quiescent. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- target protocol ---'
cat -n .agents/skills/orchestrate-story-waves/references/wave-protocol.md | sed -n '1,120p'
printf '%s\n' '--- eval assertion ---'
cat -n .agents/skills/orchestrate-story-waves/evals/evals.json | sed -n '1,80p'
printf '%s\n' '--- integration and ancestry references ---'
rg -n -C 3 'integration-phase-mode|merge-base|cherry-pick|rebase|ancestor|integration branch|integration commit|quiescent|conflict' .agents/skills/orchestrate-story-wavesRepository: agentic-workflow-kit/jig
Length of output: 13026
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- orchestration integration rules ---'
cat -n .agents/skills/orchestrate-story-waves/SKILL.md | sed -n '40,90p'
printf '%s\n' '--- all repository references to integration commands and ancestry checks ---'
rg -n -C 4 --glob '!*.lock' --glob '!node_modules/**' \
'git (merge|cherry-pick|rebase)|merge-base|is-ancestor|preserv(e|ing).*ancestor|reviewed commit|refreshed candidate' .Repository: agentic-workflow-kit/jig
Length of output: 31137
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
target = Path(".agents/skills/orchestrate-story-waves/references/wave-protocol.md").read_text()
checklist = Path("docs/delivery/greenfield/reviewer/review-checklist.md").read_text()
readme = Path("docs/delivery/greenfield/reviewer/README.md").read_text()
required = [
"preserving it as an ancestor",
"fast-forward",
"no-fast-forward",
"merge-base --is-ancestor",
]
print("target protocol:")
for phrase in required:
print(f" {phrase!r}: {phrase in target}")
print("supporting guidance:")
for phrase in required[1:]:
print(
f" {phrase!r}: "
f"checklist={phrase in checklist}, "
f"reviewer_readme={phrase in readme}"
)
print("conflict refresh rule in target:")
print(
" coordinator returns conflict to owning pair: "
"return it to the owning pair for a refreshed candidate"
in target
)
PYRepository: agentic-workflow-kit/jig
Length of output: 548
Define an ancestry-preserving integration operation.
Specify fast-forward or no-fast-forward integration and require git merge-base --is-ancestor <reviewed-commit> <integration-commit> afterward. If conflict resolution produces a new candidate, rerun the required checks and independent reviewer validation for that exact commit.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/references/wave-protocol.md around
lines 47 - 49, Update the integration protocol after PASS to explicitly require
fast-forward or no-fast-forward integration, then verify the reviewed commit
remains an ancestor of the resulting integration commit with git merge-base
--is-ancestor. If conflict resolution creates a new candidate commit, rerun all
required checks and obtain independent reviewer validation for that exact
commit.
|
|
||
| Before claiming completion, report every counted story with its thread, exact candidate, reviewer | ||
| verdict, PR or integration result, checks, merge/landing evidence, cleanup result, and the recomputed | ||
| next-ready set. Validate this skill with the open-skill validator and its eval fixtures after edits. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n '"skill_name"|evals\.get\("skill_name"\)' \
.agents/skills/orchestrate-phase-delivery/scripts/validate_evals.py \
.agents/skills/orchestrate-story-waves/evals/evals.jsonRepository: agentic-workflow-kit/jig
Length of output: 398
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- referenced skill ---'
sed -n '70,100p' .agents/skills/orchestrate-story-waves/SKILL.md
printf '%s\n' '--- validator implementation ---'
sed -n '1,90p' .agents/skills/orchestrate-phase-delivery/scripts/validate_evals.py
printf '%s\n' '--- story-wave evaluation files ---'
find .agents/skills/orchestrate-story-waves -maxdepth 3 -type f -print | sort
printf '%s\n' '--- validator references and commands ---'
rg -n -i 'validate_evals|open-skill validator|eval fixtures|orchestrate-story-waves|orchestrate-phase-delivery' \
.agents README.md 2>/dev/null | head -200Repository: agentic-workflow-kit/jig
Length of output: 6462
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- phase-delivery validator on phase-delivery fixtures ---'
python3 .agents/skills/orchestrate-phase-delivery/scripts/validate_evals.py \
.agents/skills/orchestrate-phase-delivery || true
printf '%s\n' '--- phase-delivery validator on story-waves fixtures ---'
python3 .agents/skills/orchestrate-phase-delivery/scripts/validate_evals.py \
.agents/skills/orchestrate-story-waves || true
printf '%s\n' '--- documented validation commands ---'
sed -n '20,40p' .agents/skills/orchestrate-phase-delivery/README.md
printf '%s\n' '--- available validator scripts ---'
find . -type f \( -name 'validate_skill.py' -o -name 'validate_evals.py' \) -print | sortRepository: agentic-workflow-kit/jig
Length of output: 1351
Point validation at the story-waves fixtures.
The local eval validator rejects .agents/skills/orchestrate-story-waves because it hard-codes "orchestrate-phase-delivery". Use a validator that accepts "orchestrate-story-waves" and record the exact validation command.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/orchestrate-story-waves/SKILL.md at line 89, Update the
validation step in the story-waves skill instructions to use a validator
configured to accept “orchestrate-story-waves” rather than the hard-coded
“orchestrate-phase-delivery” value, and record the exact command used to
validate the skill and its eval fixtures.
1 participant
Summary
Verification
python3 .../validate_skill.py .agents/skills/orchestrate-story-wavesgit diff --checkpnpm delivery:checkpnpm check(21/21 tasks).skillarchive outside the repositorySummary by CodeRabbit
New Features
Documentation
Tests