An additive schema version advances every open batch

CHANGELOG.md

@@ -83,6 +83,34 @@ nothing was being distributed. This is the first version that is.
 
 ### Changed
 
+- **A schema version that only widens the contract now moves every open batch onto it** (#381).
+  A batch is judged against the version it pinned at approval, and that pin used to move only
+  when somebody asked for it. Publishing an additive version now takes every batch in
+  `REPINNABLE_STATES` — `approved` or `in_annotation` — along with it, in the same transaction.
+
+  **The safety argument is a construction rather than a policy.** `diff_classes` already answers
+  *does an annotation valid under the old version stay valid under the new one?*, and when it
+  answers yes a wider contract cannot invalidate anything already drawn. A **narrowing** version
+  moves nothing, with `allow_destructive` or without it: that flag says *publish this*, never
+  *and drag every open batch across it*. `BatchService.repin` is untouched and is now the manual
+  route for exactly that case, judged against a single batch's own labels. A `completed` batch
+  never moves either way — its pin is the record of what its finished work was judged against.
+
+  This inverts a written invariant. `Batch.schema_version` said the pin "never follows the active
+  version on its own — a schema that evolved mid-batch would change the rules under work in
+  flight", and that turned out to be an argument about narrowing, which still never follows. What
+  the old rule cost was the reason it was reopened: a project could sit two versions ahead of the
+  batch somebody was annotating in, with the class they had just published invisible to them.
+
+  `SchemaService.create_version` returns `SchemaPublication` — the version, and the batches that
+  moved — instead of the version alone, and `POST /projects/{id}/schema/versions` answers
+  `{published, advanced_batches}` in place of a bare version. The reads are unchanged. The
+  annotator's add-a-class chain drops its third call, and with it the half-applied state audit
+  finding F23 was about: publishing and moving the pin are one transaction, so *the version
+  exists and the pin has not moved* is now unrepresentable rather than guarded against.
+
+  No migration, no new port method, and no event.
+
 - **The annotator's top bar has a verb for finishing a frame** (#383). Dogfooding #368's bar
   found that the commonest move in the product had no button: after annotating a frame, the
   thing to do is store it and go to the next one, and the only control that advanced was the

docs/batches.md

@@ -74,9 +74,38 @@ validated against the pinned version, not against whatever is newest.
 Approving a project that has no schema raises `SchemaNotFound`. Creating version 1 here would
 be a second door to a schema, and [schemas.md](schemas.md) has only one.
 
-## Moving the pin: `repin`
+## The pin follows a widening version on its own
 
-The pin moves only when somebody asks:
+**A version that only widens the contract moves every open batch onto it**, in the
+same transaction that publishes it (#381). `create_version` answers with the
+version *and* the batches it moved:
+
+```python
+published = schemas.create_version(project.id, [*current, LANE])
+published.published.version  # 2
+published.advanced_batches  # every batch that was `approved` or `in_annotation`
+```
+
+The safety argument is the whole rule, and it is a construction rather than a
+policy: `diff_classes` answers *does an annotation valid under the old version
+stay valid under the new one?*, and when it answers yes a wider contract cannot
+invalidate anything already drawn. So there is nothing on this path for a manual
+step to protect — and what the manual step cost was that a class published while
+somebody was annotating stayed invisible to them until they found `repin`.
+
+A **narrowing** version moves nothing, with `allow_destructive` or without it:
+that flag says *publish this*, never *and drag every open batch across it*.
+Crossing a narrowing is `repin`, one batch at a time, judged against that batch's
+own labels.
+
+`REPINNABLE_STATES` is what both routes read. A draft has no pin — approval takes
+the active version, which is the new one anyway — and a completed batch's pin is
+the record of what its finished work was judged against.
+
+## Moving the pin by hand: `repin`
+
+The pin also moves when somebody asks, which is how a narrowing version is
+crossed:
 
 ```python
 batches.repin(batch.id)  # → pinned to 3, the current active version
@@ -115,11 +144,12 @@ Re-pinning onto the version already pinned is a no-op: the same batch comes back
 written and nothing is announced. Annotations already written keep the `schema_version` they
 were stamped with - only new writes are judged against the new pin.
 
-**The caller this exists for is the annotation page.** #233's *add a class without leaving the
-job* is save → `create_version` → `repin`, in that order, and on that path the change is
-additive by construction, so the gate never fires. It fires only when somebody else narrowed
-the schema past this batch's pin in the meantime - which is the gate doing its job rather than
-getting in the way. See [ui.md](ui.md).
+**This used to be the annotation page's third call, and is not any more.** #233's *add a class
+without leaving the job* was save → `create_version` → `repin`, and on that path the change is
+additive by construction — so the version now carries the batch along and there is no third
+call to make. What is left for this method is the case the gate was always really for: somebody
+else narrowed the schema past this batch's pin, and crossing that is a decision about *this*
+batch's labels. See [ui.md](ui.md).
 
 ## The partition is exact
 

docs/schemas.md

@@ -27,7 +27,9 @@ with WorkspaceService.open("./road-signs") as workspace:
     )
     lane = LabelClass(name="lane", geometry=GeometryType.POLYGON)
 
-    schemas.create_version(project.id, [sign, lane])  # → version 1
+    published = schemas.create_version(project.id, [sign, lane])
+    published.published.version  # 1
+    published.advanced_batches  # the open batches this version took with it
 
     schemas.get_active(project.id)  # the highest version
     schemas.get(project.id, 1)  # any version, forever
@@ -231,10 +233,27 @@ version's classes are bound to. It is what an annotation's `geometry.type` is
 membership-tested against - the union's discriminator values *are* `GeometryType` members,
 so nothing translates in between.
 
+## Publishing catches the open batches up
+
+A batch is judged against the version it pinned at approval, and that pin used to move only
+when somebody asked. **A version that only widens the contract now takes every open batch with
+it**, in the same transaction that publishes it (#381), and `create_version` answers with the
+batches it moved so a surface can say so rather than leave it to be discovered.
+
+The whole safety argument is the section below: additive means every annotation valid under the
+old version is valid under the new one, so a wider contract cannot invalidate anything already
+drawn. A **narrowing** version moves nothing, `allow_destructive` or not — that flag says
+*publish this*, never *and drag every open batch across it* — and crossing one is
+`BatchService.repin`, judged against a single batch's own labels. See
+[batches.md](batches.md).
+
+Publishing the contract already in force writes nothing and therefore moves nothing: an
+operation that writes nothing cannot have an effect.
+
 ## Additive versus destructive
 
-One question draws the line: **does an annotation that was valid under the previous version
-stay valid under this one?**
+One question draws the line, and it is the same question the paragraph above rests on: **does an
+annotation that was valid under the previous version stay valid under this one?**
 
 | Change | Kind |
 | --- | --- |

examples/http_end_to_end.py

@@ -315,7 +315,10 @@ def _walk(client: Client, base_url: str, downloads: Path) -> Summary:
     """Ingest to an exported release, one request at a time."""
     # (1) A project, and the labeling contract its work will be judged against.
     project = client.json("POST", "/projects", 201, json_body={"name": "chest-xray"})["id"]
-    schema = client.json(
+    # The answer is a *publication*: the version, plus every open batch that moved
+    # onto it. There are none yet — the project has no batch at all — and empty is
+    # the ordinary answer rather than a failure.
+    published = client.json(
         "POST",
         f"/projects/{project}/schema/versions",
         201,
@@ -329,6 +332,7 @@ def _walk(client: Client, base_url: str, downloads: Path) -> Summary:
             ]
         },
     )
+    schema = published["published"]
     _say(f"project {project} with schema v{schema['version']}")
 
     # (2) Offer it some data. Registration is upload-only: an HTTP client has

examples/ingest_end_to_end.py

@@ -213,7 +213,7 @@ def main(dest: Path) -> Summary:
         # (1) A project, its 1:1 dataset, and a labeling contract. Nothing here
         # writes a label, but a batch cannot be approved without a schema to pin.
         project = projects.create("dashcam", description="Ingest end-to-end demo")
-        schema = schemas.create_version(project.id, CLASSES)
+        schema = schemas.create_version(project.id, CLASSES).published
         _say(f"project {project.name!r} ({project.id}) with schema v{schema.version}")
 
         # (2) Ten seconds of video, registered as an origin. The extraction rate

examples/mcp_end_to_end.py

@@ -227,7 +227,9 @@ async def tool(name: str, /, **arguments: Any) -> CallToolResult:
         # (2) Declare the contract before any work is judged against it. The
         # domain models go straight into the tool signature, so these two dicts
         # are validated by the kernel's own rules and refused in its own words.
-        schema = ok(
+        # A publication: the version, and the open batches it took with it. None
+        # here — the project has no batch yet — and empty is the ordinary answer.
+        published = ok(
             await tool(
                 "create_schema_version",
                 project=PROJECT,
@@ -237,6 +239,7 @@ async def tool(name: str, /, **arguments: Any) -> CallToolResult:
                 ],
             )
         )
+        schema = published["published"]
         assert ok(await tool("get_schema", project=PROJECT))["active_version"] == schema["version"]
 
         # (3) Read the folder in. **One call, and it returns when the work is

examples/sdk_end_to_end.py

@@ -288,7 +288,7 @@ def main(dest: Path) -> Summary:
 
         # (3) Version 1 of the labeling contract. Versions are 1..N and are never
         # edited; "active" is derived (the highest), never a stored column.
-        schema = schemas.create_version(project.id, CLASSES)
+        schema = schemas.create_version(project.id, CLASSES).published
         _say(f"schema v{schema.version}: {', '.join(c.name for c in schema.classes)}")
 
         # (4) Six generated frames on disk, and the directory holding them

frontend/app/cycle/cycle.spec.ts

@@ -549,6 +549,96 @@ test("the whole cycle, from opening the app to a downloaded export", async ({ pa
     // Back to select, so the rest of the walk starts where it used to.
     await page.getByTestId("tool-select").click();
 
+    /*
+     * 3a-bis — #381: **a version published anywhere moves this batch's pin, and
+     * nobody presses re-pin.**
+     *
+     * This is the only run that can show it. The unit suites stub the publish, so
+     * they can assert what the client sent and rendered; whether the *server*
+     * moved the pin in the same transaction is a fact about `SchemaService`, and
+     * this is the one place that service is real. Before #381 the pin stayed at
+     * v1 here and the new class was invisible in this batch until somebody found
+     * the re-pin — which is the dead end the issue was reopened for.
+     *
+     * Published through the API rather than the Schema tab because the claim is
+     * about the kernel, not about a screen: leaving the editor and coming back
+     * would test navigation as well, and the Schema tab publishing *at all* is
+     * already covered above. The extra class is never drawn, and
+     * `DatasetStats.per_class` lists only classes with annotations, so nothing
+     * downstream counts it.
+     */
+    const job = await page.request.get(`${origin}/jobs/${jobId}`, {
+      headers: { Authorization: `Bearer ${token()}` },
+    });
+    const batchId = (await job.json()).batch_id;
+    const batch = await page.request.get(`${origin}/batches/${batchId}`, {
+      headers: { Authorization: `Bearer ${token()}` },
+    });
+    const beforePin = (await batch.json()).schema_version;
+    const projectId = (await batch.json()).project_id;
+
+    // The whole contract plus one — `create_version` takes the entire class list,
+    // so a class left out is a class removed, and reading the active version is
+    // how this stays an *additive* change rather than an accidental narrowing.
+    const active = await page.request.get(`${origin}/projects/${projectId}/schema`, {
+      headers: { Authorization: `Bearer ${token()}` },
+    });
+    const current = (await active.json()).classes;
+
+    const grown = await page.request.post(`${origin}/projects/${projectId}/schema/versions`, {
+      headers: { Authorization: `Bearer ${token()}` },
+      data: {
+        classes: [...current, { name: "pedestrian", geometry: "bbox" }],
+        provenance: "curated",
+      },
+    });
+    expect(grown.status()).toBe(201);
+    const publication = await grown.json();
+    expect(publication.published.version).toBe(beforePin + 1);
+    // The response names what it moved, which is what stops a publish being a
+    // silent side effect.
+    expect(publication.advanced_batches).toContain(batchId);
+
+    await page.reload();
+    await expect(page.getByTestId("annotation-page")).toBeVisible();
+    await expect(page.getByTestId("pinned-schema")).toHaveText(`v${beforePin + 1}`);
+    // And the class it brought is drawable here, which is the whole point.
+    await expect(page.getByTestId("class-row-pedestrian")).toBeVisible();
+
+    /*
+     * 3a-ter — **the annotator's own add-a-class door, which is now two calls.**
+     *
+     * `runAddClass` was save → publish → re-pin, and #381 took the third away: the
+     * publish moves the pin itself. Nothing anywhere exercised this door in a
+     * browser — `annotate.spec.ts` only asserts it is *absent* in read-only mode,
+     * and the demo it runs against has no project behind it — so its whole
+     * coverage was unit tests against stubs. That is the shape of gap that hid the
+     * per-batch diff defect, which is why it is closed here rather than argued
+     * about.
+     *
+     * The request log is the assertion that matters: a `/repin` call would mean
+     * the step is still being made by the client.
+     */
+    const posted: string[] = [];
+    const record = (request: import("@playwright/test").Request): void => {
+      if (request.method() === "POST") posted.push(new URL(request.url()).pathname);
+    };
+    page.on("request", record);
+
+    await page.getByTestId("tool-add-class").click();
+    await expect(page.getByTestId("add-class-dialog")).toBeVisible();
+    await page.getByTestId("class-name-new").fill("cyclist");
+    await page.getByTestId("add-class-submit").click();
+    await expect(page.getByTestId("add-class-dialog")).toHaveCount(0);
+    await expect(page.getByTestId("class-row-cyclist")).toBeVisible();
+    page.off("request", record);
+
+    expect(posted.filter((path) => path.endsWith("/schema/versions"))).toHaveLength(1);
+    expect(posted.filter((path) => path.endsWith("/repin"))).toHaveLength(0);
+    // The pin followed anyway, which is the whole of what the third call used to
+    // do — and the class the dialog armed is drawable on this frame.
+    await expect(page.getByTestId("pinned-schema")).toHaveText(`v${beforePin + 2}`);
+
     // 3b — the review round-trip, on the frame we are already standing on.
     //
     // **This is the half of the progress machine that had no door** (audit F24):
@@ -847,10 +937,13 @@ test("the whole cycle, from opening the app to a downloaded export", async ({ pa
     await page.getByTestId("approve-submit").click();
     await expect(page.getByTestId(`state-${CORRECTION}`)).toHaveText("approved");
     // The child pins the project's *active* version at its own approval rather
-    // than inheriting the parent's. They are the same number here because
-    // nothing has published since — the claim is that it pinned, not that it
-    // copied.
-    await expect(page.getByTestId(`batch-${CORRECTION}`)).toContainText("v1");
+    // than inheriting the parent's. They are the same number here — **v3**, after
+    // the two additive publishes above, both of which moved the parent onto them
+    // as well — so the claim this makes is that it pinned, not that it copied.
+    // Distinguishing the two needs a parent that is *behind*, which only a
+    // narrowing version can produce, and that belongs to the kernel suite rather
+    // than to a walk through the app.
+    await expect(page.getByTestId(`batch-${CORRECTION}`)).toContainText("v3");
     await page.getByTestId(`start-${CORRECTION}`).click();
     await expect(page.getByTestId(`state-${CORRECTION}`)).toHaveText("in progress");