Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
14a8d2f
feat(workflows): add standalone WorkflowResolver and overlay subsystem
Jul 16, 2026
01e0455
fix(workflows): reject symlinked overlay directories in layer sources
Jul 16, 2026
c70a5d6
fix(workflows): address Copilot review findings in merge engine
Jul 16, 2026
cc28185
refactor(workflows): simplify overlay architecture to 2-tier
Jul 16, 2026
d51241b
fix(workflows): harden overlay symlink handling
Jul 17, 2026
ed0daa7
docs(workflows): remove stale installed-overlay references from workf…
Jul 17, 2026
b890fdd
fix(overlays): detect ancestor-conflict anchors in merge_steps
Jul 17, 2026
aa4eb4d
Merge main into feat/workflow-overlays
Jul 17, 2026
586500f
fix(overlays): fix over-broad conflict detection and non-deterministi…
Jul 17, 2026
d909024
fix(overlays): reject ID trailing newlines and reuse existing .yaml path
Jul 17, 2026
c1e0683
fix(overlays): fix display order inversion and wrap file-read errors
Jul 17, 2026
2d5b143
fix: rename misleading overlay test
Jul 17, 2026
0a6dc13
fix: remove EOF blank line in overlay resolver
Jul 17, 2026
23f0890
Merge remote-tracking branch 'origin/main' into feat/workflow-overlays
Jul 17, 2026
b4bb6f8
Potential fix for pull request finding
markuswondrak Jul 17, 2026
21dda03
fix: handle overlay read and enumeration errors
Jul 17, 2026
0969d9f
fix(overlays): validate resolver workflow IDs
Jul 17, 2026
010fb67
Potential fix for pull request finding
markuswondrak Jul 17, 2026
28c85ff
fix(overlays): drop _remove_sources_recursively from remove branch
Jul 17, 2026
9525d79
Merge branch 'feat/workflow-overlays' of https://github.com/markuswon…
Jul 17, 2026
9e6aa65
Fix CLI overlay ID validation anchoring
Jul 17, 2026
b9de204
fix(workflows): validate workflow_id in layer sources before path con…
Jul 17, 2026
69301a6
fix(workflows): align layer source validation with _safe_workflow_id_dir
Jul 18, 2026
9ebfc3a
Potential fix for pull request finding
markuswondrak Jul 18, 2026
01c8505
Merge branch 'feat/workflow-overlays' of https://github.com/markuswon…
Jul 18, 2026
e70bbb7
fix(overlays): resolve identity from manifest field, not filename
Jul 18, 2026
1e72c4e
Merge branch 'main' into feat/workflow-overlays
Jul 18, 2026
79c5b93
Potential fix for pull request finding
markuswondrak Jul 18, 2026
901a7c8
fix(workflows): make validation behavior consistent across YAML loadi…
Jul 18, 2026
f4dd3fe
Merge branch 'feat/workflow-overlays' of https://github.com/markuswon…
Jul 18, 2026
1e44f13
fix(overlays): list disabled overlays in management view
Jul 18, 2026
fc9d710
Potential fix for pull request finding
markuswondrak Jul 18, 2026
08fa12e
docs: align overlay extends and resolver contract
Jul 18, 2026
54c5b9f
fix: use atomic write for overlay file updates to prevent hard-link a…
Jul 18, 2026
3f047c9
fix(composer): preserve invalid base definition instead of coercing s…
Jul 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
186 changes: 185 additions & 1 deletion docs/reference/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,192 @@ specify workflow add <source>
| `--dev` | Install from a local workflow YAML file or directory |
| `--from <url>` | Install from a custom URL (`<source>` names the expected workflow ID) |

Installs a workflow from the catalog, a URL (HTTPS required), or a local file path.
Installs a workflow from the catalog, a URL (HTTPS required), a local YAML file, or a local directory containing `workflow.yml`.

## Workflow Overlays

Workflow overlays let a project extend or override an installed workflow without editing the installed `workflow.yml`. This keeps local customizations safe across `specify bundle update` or `specify workflow add` upgrades.

When `specify workflow run <workflow-id>` loads a workflow, the engine composes the base workflow with all enabled overlays for that workflow id. The result is validated like any other workflow definition.

### How Overlays Work

An overlay is a YAML file that declares a set of edit operations against the step list of a base workflow. Overlays are applied in priority order (lowest first, highest last); equal-priority overlays are applied in source order (last applied wins).

Project overlay files live at:

| Location | Purpose |
| --- | --- |
| `.specify/workflows/overlays/<id>/*.yml` | Project-local customizations |

### Overlay File Format

The recommended edit format uses the operation name as the key and the anchor step id as the value:

```yaml
id: "my-overlay"
extends: "speckit"
priority: 10
enabled: true
edits:
- insert_after: implement
step:
id: run-lint
type: shell
run: "ruff check src/"

- replace: review-spec
step:
id: review-spec
type: gate
message: "Review the generated spec (overlay override)."
options: [approve, reject]
on_reject: abort
```

The explicit form is also supported:

```yaml
edits:
- operation: insert_after
anchor: implement
step:
id: run-lint
type: shell
run: "ruff check src/"
```

#### Fields

| Field | Required | Description |
| --- | --- | --- |
| `id` | yes | Identifier for this overlay. Used in `specify workflow overlay *` commands. Must be lowercase letters, digits, and hyphens only; no dots, underscores, path separators, or `overlays`. |
| `extends` | yes | The workflow id this overlay applies to. Uses the same safe-id format as `id`; `overlays`, `runs`, and `steps` are reserved. |
| `priority` | yes | Integer `>= 1`. Higher priority overlays are applied later and win conflicts. |
| `enabled` | no | Boolean. Defaults to `true`. Disabled overlays are ignored. |
| `edits` | yes | Non-empty list of edit operations. |

#### Edit Operations

| Operation | `step` required | Effect |
| --- | --- | --- |
| `insert_after` | yes | Insert `step` immediately after the anchor step. |
| `insert_before` | yes | Insert `step` immediately before the anchor step. |
| `replace` | yes | Replace the anchor step with `step`. |
| `remove` | no | Remove the anchor step from the list. |

The `anchor` is the `id` of a step in the base workflow. Anchors are resolved recursively inside `then`, `else`, `steps`, `cases.*`, and `default` blocks, so nested base steps can also be targeted. Fan-out templates (`step` inside a `fan-out` step) are **not** valid anchors.

Step ids must not contain `:` — that character is reserved for engine-generated nested ids.

### Overlay CLI Commands

#### Add a Project Overlay

```bash
specify workflow overlay add <path-to-overlay.yml> --priority <n>
```

Validates the overlay file and copies it to `.specify/workflows/overlays/<extends>/<id>.yml`. `--priority` overrides the `priority` field in the file.

#### List Overlays

```bash
specify workflow overlay list <workflow-id>
```

Shows all overlays for the workflow, ordered by resolver precedence. Disabled overlays are marked as disabled in the listing and are ignored during workflow resolution.

#### Change Priority

```bash
specify workflow overlay set-priority <workflow-id> <overlay-id> <n>
```

#### Enable or Disable

```bash
specify workflow overlay disable <workflow-id> <overlay-id>
specify workflow overlay enable <workflow-id> <overlay-id>
```

#### Remove

```bash
specify workflow overlay remove <workflow-id> <overlay-id>
```

Removes the project overlay file.

#### Inspect the Composed Workflow

```bash
specify workflow resolve <workflow-id>
```

Prints the layer stack (base + overlays) and the source attribution for each step after composition. Useful for debugging which overlay contributed or overrode a step.

### Example: Adding Automated Linting after Implementation

Given the built-in `speckit` workflow, create `project-overlay.yml`:

```yaml
id: "add-lint"
extends: "speckit"
priority: 10
edits:
- insert_after: implement
step:
id: run-lint
type: shell
run: "ruff check src/"
```

Install it:

```bash
specify workflow overlay add project-overlay.yml --priority 10
```

Run the workflow:

```bash
specify workflow run speckit -i spec="Build a kanban board"
```

The composed workflow will now run the full SDD cycle and execute `ruff check src/` automatically after the `implement` step.

### Example: Replacing a Gate

```yaml
id: "skip-plan-review"
extends: "speckit"
priority: 20
edits:
- replace: review-plan
step:
id: review-plan
type: command
command: speckit.plan
input:
args: "{{ inputs.spec }}"
```

Higher priority (`20`) means this overlay is applied after the `add-lint` overlay above. It replaces the `review-plan` gate with a non-interactive command.

### Interaction with Bundles and Updates

`specify workflow add <local-directory>` installs `workflow.yml` from the local directory into `.specify/workflows/<id>/`.

When an installed workflow is refreshed or reinstalled, project overlays in `.specify/workflows/overlays/<id>/` are preserved because they live outside the installed workflow directory.

### Limitations

- Overlays operate on the step list only. They cannot change workflow metadata (name, description, inputs, `requires`) or expression logic.
- Fan-out templates cannot be used as anchors.
- An overlay that targets a step id that does not exist in the base workflow will raise a validation error when the workflow is resolved.
- Overlays cannot target steps added by other overlays.
- Overlays cannot add new inputs or change the input schema of the base workflow.
## Update Workflows

```bash
Expand Down
108 changes: 107 additions & 1 deletion src/specify_cli/workflows/_commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,13 @@
)
workflow_step_app.add_typer(workflow_step_catalog_app, name="catalog")

workflow_overlay_app = typer.Typer(
name="overlay",
help="Manage workflow overlays",
add_completion=False,
)
workflow_app.add_typer(workflow_overlay_app, name="overlay")


def _error_console(json_output: bool):
"""Console for error text: stderr under ``--json`` so the JSON stdout
Expand Down Expand Up @@ -192,6 +199,10 @@ def _reject_unsafe_workflow_storage(project_root: Path) -> None:
project_root / ".specify" / "workflows" / "runs",
".specify/workflows/runs",
)
_reject_unsafe_dir(
project_root / ".specify" / "workflows" / "overlays",
".specify/workflows/overlays",
)
Comment thread
markuswondrak marked this conversation as resolved.


def _scan_for_workflow_owner(parts: tuple[str, ...]) -> int | None:
Expand Down Expand Up @@ -366,7 +377,7 @@ def ownership_for(candidate: Path) -> tuple[Path, str] | None:


_WORKFLOW_ID_PATTERN = re.compile(r"^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$")
_RESERVED_WORKFLOW_IDS: frozenset[str] = frozenset({"runs", "steps"})
_RESERVED_WORKFLOW_IDS: frozenset[str] = frozenset({"overlays", "runs", "steps"})


def _reject_insecure_download_redirect(old_url: str, new_url: str) -> None:
Expand Down Expand Up @@ -2386,6 +2397,9 @@ def workflow_info(
# Local workflow definition not found on disk; fall back to
# catalog/registry lookup below.
pass
except ValueError as exc:
console.print(f"[red]Error:[/red] Invalid workflow: {_escape_markup(str(exc))}")
raise typer.Exit(1)

if definition:
console.print(f"\n[bold cyan]{definition.name}[/bold cyan] ({definition.id})")
Expand Down Expand Up @@ -3152,6 +3166,98 @@ def workflow_step_catalog_remove(
console.print(f"[green]✓[/green] Step catalog source '{removed_name}' removed")


@workflow_overlay_app.command("add")
def workflow_overlay_add_cmd(
source: Path = typer.Argument(..., help="Path to overlay YAML file"),
priority: int | None = typer.Option(
None, "--priority", help="Overlay priority (higher wins, >= 1)"
),
):
"""Add a project-local overlay for a workflow."""
from .overlays._commands import workflow_overlay_add

project_root = _require_specify_project()
if workflow_overlay_add(project_root, source, priority) is None:
raise typer.Exit(1)


@workflow_overlay_app.command("set-priority")
def workflow_overlay_set_priority_cmd(
workflow_id: str = typer.Argument(..., help="Workflow ID the overlay extends"),
overlay_id: str = typer.Argument(..., help="Overlay ID"),
priority: int = typer.Argument(..., help="New priority (>= 1)"),
):
"""Set the priority of a project-local overlay."""
from .overlays._commands import workflow_overlay_set_priority

project_root = _require_specify_project()
if not workflow_overlay_set_priority(project_root, workflow_id, overlay_id, priority):
raise typer.Exit(1)


@workflow_overlay_app.command("enable")
def workflow_overlay_enable_cmd(
workflow_id: str = typer.Argument(..., help="Workflow ID the overlay extends"),
overlay_id: str = typer.Argument(..., help="Overlay ID"),
):
"""Enable a project-local overlay."""
from .overlays._commands import workflow_overlay_enable

project_root = _require_specify_project()
if not workflow_overlay_enable(project_root, workflow_id, overlay_id):
raise typer.Exit(1)


@workflow_overlay_app.command("disable")
def workflow_overlay_disable_cmd(
workflow_id: str = typer.Argument(..., help="Workflow ID the overlay extends"),
overlay_id: str = typer.Argument(..., help="Overlay ID"),
):
"""Disable a project-local overlay."""
from .overlays._commands import workflow_overlay_disable

project_root = _require_specify_project()
if not workflow_overlay_disable(project_root, workflow_id, overlay_id):
raise typer.Exit(1)


@workflow_overlay_app.command("remove")
def workflow_overlay_remove_cmd(
workflow_id: str = typer.Argument(..., help="Workflow ID the overlay extends"),
overlay_id: str = typer.Argument(..., help="Overlay ID"),
):
"""Remove a project-local overlay."""
from .overlays._commands import workflow_overlay_remove

project_root = _require_specify_project()
if not workflow_overlay_remove(project_root, workflow_id, overlay_id):
raise typer.Exit(1)


@workflow_overlay_app.command("list")
def workflow_overlay_list_cmd(
workflow_id: str = typer.Argument(..., help="Workflow ID"),
):
"""List overlays for a workflow."""
from .overlays._commands import workflow_overlay_list

project_root = _require_specify_project()
if workflow_overlay_list(project_root, workflow_id) is None:
raise typer.Exit(1)


@workflow_app.command("resolve")
def workflow_resolve_cmd(
workflow_id: str = typer.Argument(..., help="Workflow ID to resolve"),
):
"""Show layer attribution for a resolved workflow."""
from .overlays._commands import workflow_resolve

project_root = _require_specify_project()
if workflow_resolve(project_root, workflow_id) is None:
raise typer.Exit(1)


def register(app: typer.Typer) -> None:
"""Attach the workflow command group to the root Typer app."""
app.add_typer(workflow_app, name="workflow")
25 changes: 22 additions & 3 deletions src/specify_cli/workflows/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,11 @@ def __init__(self, data: dict[str, Any], source_path: Path | None = None) -> Non
def from_yaml(cls, path: Path) -> WorkflowDefinition:
"""Load a workflow definition from a YAML file."""
with open(path, encoding="utf-8") as f:
data = yaml.safe_load(f)
try:
data = yaml.safe_load(f)
except yaml.YAMLError as exc:
msg = f"Invalid YAML in {path}: {exc}"
raise ValueError(msg) from exc
if not isinstance(data, dict):
msg = f"Workflow YAML must be a mapping, got {type(data).__name__}."
raise ValueError(msg)
Expand All @@ -88,7 +92,11 @@ def from_yaml(cls, path: Path) -> WorkflowDefinition:
@classmethod
def from_string(cls, content: str) -> WorkflowDefinition:
"""Load a workflow definition from a YAML string."""
data = yaml.safe_load(content)
try:
data = yaml.safe_load(content)
except yaml.YAMLError as exc:
msg = f"Invalid YAML: {exc}"
raise ValueError(msg) from exc
if not isinstance(data, dict):
msg = f"Workflow YAML must be a mapping, got {type(data).__name__}."
raise ValueError(msg)
Expand Down Expand Up @@ -727,13 +735,24 @@ def load_workflow(self, source: str | Path) -> WorkflowDefinition:
ValueError:
If the workflow YAML is invalid.
"""
from .overlays import WorkflowResolver

path = Path(source).expanduser()

# Try as a direct file path first
if path.suffix.lower() in (".yml", ".yaml") and path.is_file():
return WorkflowDefinition.from_yaml(path)

# Try as an installed workflow ID
# Try as an installed workflow ID, resolving any overlays.
resolver = WorkflowResolver(self.project_root)
try:
return resolver.resolve(str(source))
Comment thread
markuswondrak marked this conversation as resolved.
except FileNotFoundError:
# Fall back to the direct workflow.yml path so callers still get
# the original error when the workflow id is not installed.
pass

# Legacy direct path check for workflows installed without registry entries.
installed_path = (
self.project_root
/ ".specify"
Expand Down
Loading