[ESSREDUCE] Minimal implementation-independent workflow specifications - #690
Open
SimonHeybrock wants to merge 1 commit into
Open
[ESSREDUCE] Minimal implementation-independent workflow specifications#690SimonHeybrock wants to merge 1 commit into
SimonHeybrock wants to merge 1 commit into
Conversation
WorkflowSpec describes a workflow's user-facing interface (identity, title/description, one pydantic params model, structural output descriptions) without factories, sciline keys, or registries, so generic UIs can be generated from it regardless of where compute happens. serialize() projects one-way onto SerializedWorkflowSpec (params as JSON Schema) for cross-process consumers; authoritative validation stays with the process owning the model class. Includes a scipp-free shared parameter vocabulary (unit enums, range/edges models with cross-field validation) with scipp conversions quarantined in spec.conversions, and ADR 0001 recording the design and its rationale (see #653, scipp/esslivedata#889). Adds pydantic as an essreduce dependency. The existing ess.reduce.parameter/workflow machinery is superseded but untouched; removal is a later hard break. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fresh counter-proposal for #653, as announced in the review on #689 — I propose closing that PR in favor of this one.
The core difference to #689: the spec is pure interface. No workflow factory, no sciline keys, no
use_workflow_default— nothing that presumes a particular implementation or execution location. The same spec must make sense whether the workflow runs as a local sciline pipeline, behind a web service, or as a cluster job; compute binding is a parallel mechanism, deliberately out of scope here.Main decisions (full rationale in ADR 0001, included in this PR):
Parameterentries. Enables cross-parameter validation and JSON Schema for free. The graph-derived "select outputs → see relevant params" feature is dropped, not ported: it treats output selection as workflow slicing, which only the sciline implementation can express.titleanddescriptionmandatory; identity isname+version— noinstrument, noWorkflowIdclass. Scoping and global uniqueness are the enumerating context's responsibility.WorkflowSpec(in-process, holds the params model class, full validation) andSerializedWorkflowSpec(plain data, params as JSON Schema, JSON-round-trippable). No inverse — validators don't survive JSON Schema, so authoritative validation stays with the process owning the model class; remote consumers validate optimistically. This matches the announcement-as-contract design from Rethink workflow registration: move schema ownership to domain packages esslivedata#889.ArraySpec(dims, unit, coord units) — plain data, no scipp, replacingsc.DataArraydefault-factory templates. Scalars with units are the 0-d case.spec.conversions. Value defaults are set by workflow authors at the use site, not by the vocabulary.ess.livedataWorkflowSpeceventually inherits from this, gated on its plannedArraySpecmigration; base field names are a strict subset of livedata's for that reason.Adds
pydantic>=2.5as an essreduce dependency (already present in the locked CI environments). The existingess.reduce.parameter/workflow/widgets machinery is superseded but untouched; removal is a later hard break — no one depends on it currently.🤖 Generated with Claude Code