Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ GET /batches/{batch_id}
DELETE /batches/{batch_id} ?confirm=true
POST /batches/{batch_id}/approve with a partition spec
POST /batches/{batch_id}/start
GET /batches/{batch_id}/pre-label the classes a run would ask for
POST /batches/{batch_id}/pre-label launch, poll the job
POST /batches/{batch_id}/repin ?allow_destructive=
POST /batches/{batch_id}/complete
Expand Down Expand Up @@ -776,9 +777,10 @@ each is a decision somebody will otherwise try to "fix":
- **Unknown keys pass.** `additionalProperties: false` constrains what the API *accepts*,
not what it may one day *send*. A client that refused an added field would turn every
backward-compatible release into a broken page.
- **An unknown member of an *open* vocabulary passes.** Six vocabularies carry
`x-visionset-open` in the spec — the four `allowed_actions` sets, `capabilities`, and
`SuggestionOut.parameters` — and the generated check for one accepts a member this client
- **An unknown member of an *open* vocabulary passes.** Seven vocabularies carry
`x-visionset-open` in the spec — the four `allowed_actions` sets, `capabilities`,
`SuggestionOut.parameters`, and the reasons a class is left out of a pre-label prompt — and
the generated check for one accepts a member this client
never compiled against, exactly as it accepts an added field. Every other enum still
refuses, and refuses the whole response with it: a value the client must *switch* on has no
honest rendering to fall back to. The line between them is the field's shape. A vocabulary
Expand Down
13 changes: 13 additions & 0 deletions docs/batches.md
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,19 @@ no overlap with a measured asset is discarded separately, and `regions_out_of_bo
many; unmeasured assets remain eligible. A schema with no such class is refused up front; see
[inference.md](inference.md#what-a-connection-can-be-asked-for).

**A class is left out of the prompt for either of two reasons, and both are published.** It does
not admit `bbox`, so a detection has no shape to land as; or it declares a required attribute,
which a bare prediction carries no value for. Neither is visible in a run's outcome - a schema
whose `vehicle` requires a `color` completes a run, labels no vehicles, and says nothing about
why - so `GET /batches/{id}/pre-label` answers both halves before a run starts: `asked_classes`
is the prompt, and `excluded_classes` names the rest with every reason that holds against each.
Every class the pinned schema declares appears in exactly one of the two lists. It is derived
from the schema alone and needs no connection, so a dialog can name the classes before anybody
has chosen a model; a batch whose schema has no askable class at all is refused with the same
`SCHEMA_HAS_NO_DETECTABLE_CLASS` the launch answers, rather than reported as an empty prompt. At
a terminal `visionset batch pre-label` writes the same two lines to stderr before the first
forward pass.

**What lands enters at `pre_labeled`, never `annotated`.** Nobody judged it, so it arrives in its
own editable state rather than claiming to be somebody's work - see
[annotations.md](annotations.md#provenance-is-the-models-own-rule-not-the-services). It is not
Expand Down
7 changes: 5 additions & 2 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,8 +358,11 @@ interrupted run. The batch id goes to stdout.
`DatasetService.promote` - it takes a *batch* id and derives the dataset, which is why it lives here.

`pre-label BATCH_ID CONNECTION [--minimum-confidence FLOAT]` blocks and calls
`visionset.inference.pre_label` inline because a terminal has no dispatcher. Progress and the
summary are written to stderr; normal stdout contains `annotations_written`. With `--json`, the
`visionset.inference.pre_label` inline because a terminal has no dispatcher. Before the first
forward pass it names the classes it is about to ask for, and every class of the pinned schema it
is leaving out with the reason - a class the prompt omits labels nothing, and afterwards there is
only the silence to explain. Progress and the summary are written to stderr; normal stdout
contains `annotations_written`. With `--json`, the
command prints the complete outcome instead, including `regions_discarded` for unmappable model
labels and `regions_out_of_bounds` for mapped regions without overlap with a measured asset.

Expand Down
11 changes: 11 additions & 0 deletions docs/ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -750,6 +750,17 @@ asset already pre-labeled, annotated, skipped, awaiting review or accepted is pa
label that lands enters at `pre_labeled`, never `annotated`, so an annotator corrects a machine's
guess rather than inheriting it silently as their own work.

**The prompt is named, and so is everything left out of it.** A count of assets says nothing about
which classes a run will look for, so a schema whose `vehicle` requires an attribute completes a
run, labels no vehicles, and reads exactly like a run that should have labeled something. The
dialog reads `GET /batches/{id}/pre-label` when it opens and shows both halves: the classes it
asks for, and beside them each class it does not, with the reason - no box, or an attribute a
prediction cannot supply. The lists come off the wire rather than being derived from the pinned
schema in the browser, because the same narrowing decides what the run really prompts with. They
are shown again under a settled run's summary, which is where a run that labeled nothing is
actually read. A schema with no askable class at all refuses this read, and the dialog renders
that refusal and leaves `Start` dead rather than waiting for the press to produce it.

The route answers `202` with a background job, on the export and weight-download routes'
contract, and the dialog polls it exactly as `ExportDialog` polls an export: nothing here waits
for the run to finish, but nothing closes over an outcome unseen either. Every refusal the route
Expand Down
154 changes: 153 additions & 1 deletion frontend/ui-core/src/generated/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,30 @@ export interface paths {
path?: never;
cookie?: never;
};
get?: never;
/**
* Pre Label Plan
* @description The classes a pre-labeling run over this batch would ask a model for.
*
* A run's prompt is the batch's pinned schema narrowed to the classes a bare
* box prediction can be written as, and that narrowing is invisible once the
* run has finished: a schema whose `vehicle` class requires an attribute
* yields no vehicles and says nothing about why. Read this before launching to
* say which classes are in the prompt and which are not, with the reason
* beside each one.
*
* Derived, never stored, and free of the connection the launch needs — the
* prompt is a property of the schema alone, so this answers the same lists
* whichever model is about to be asked.
*
* A batch that no run could touch is refused rather than answered with empty
* lists, on the same terms the launch itself uses: an unknown batch is 404
* `BATCH_NOT_FOUND`, a batch that is not `in_annotation` is 409
* `BATCH_NOT_IN_ANNOTATION`, and a pinned schema with no class a box can be
* written as is 409 `SCHEMA_HAS_NO_DETECTABLE_CLASS`. A batch open for
* annotation but pinning no schema version is a broken invariant and answers
* 500 `WORKSPACE_CORRUPT`.
*/
get: operations["pre_label_plan"];
put?: never;
/**
* Pre Label Batch
Expand Down Expand Up @@ -3936,6 +3959,58 @@ export interface components {
*/
type: "polyline";
};
/**
* PreLabelExclusionOut
* @description A class in a batch's pinned schema that a pre-labeling run will not ask for.
*
* Both reasons are properties of the class as the schema declares it, so the
* remedy is a schema edit: give the class `bbox` among its geometries, or drop
* the `required` flag from the attribute a prediction cannot supply.
*
* `reasons` can carry both at once, and every reason that holds is listed —
* a class told only that it admits no box, then given one, would otherwise
* stay silently absent from the next run's prompt.
*/
PreLabelExclusionOut: {
/** Name */
name: string;
/** Reasons */
reasons: components["schemas"]["PreLabelExclusionReason"][];
};
/**
* PreLabelExclusionReason
* @description Why a schema's class is not among the words a run asks for.
*
* Open because it travels as a list a client renders member by member rather
* than switches on: a release that finds a third way a class cannot hold a
* detection must not cost an older client the whole plan, and the class it
* names is visibly left out whether or not that client can word the reason.
* @enum {string}
*/
PreLabelExclusionReason: "no_bbox_geometry" | "required_attribute" | (string & {});
/**
* PreLabelPlanOut
* @description The words a pre-labeling run over this batch would ask a model for.
*
* A run's prompt is the batch's pinned schema, narrowed to the classes a bare
* box prediction can be written as. That narrowing is invisible in the run's
* result — a schema whose `vehicle` class requires a `color` attribute yields
* no vehicles and no explanation — so it is published here, before a run
* starts, with the left-out classes named beside the asked-for ones.
*
* Every class the pinned schema declares appears in exactly one of the two
* lists, both in the schema's own declaration order. A batch whose schema has
* no askable class at all is refused rather than answered with an empty
* `asked_classes`: pre-labeling it is impossible, not merely unproductive.
*/
PreLabelPlanOut: {
/** Asked Classes */
asked_classes: string[];
/** Excluded Classes */
excluded_classes: components["schemas"]["PreLabelExclusionOut"][];
/** Schema Version */
schema_version: number;
};
/**
* PreLabelRequest
* @description Which model should pre-label this batch, and how sure it has to be.
Expand Down Expand Up @@ -5673,6 +5748,82 @@ export interface operations {
};
};
};
pre_label_plan: {
parameters: {
query?: never;
header?: never;
path: {
batch_id: string;
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Successful Response */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["PreLabelPlanOut"];
};
};
/** @description Missing or invalid bearer token */
401: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorBody"];
};
};
/** @description No such resource */
404: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorBody"];
};
};
/** @description The resource's state refuses this request */
409: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorBody"];
};
};
/** @description The request payload is not processable */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorBody"];
};
};
/** @description Unhandled server error, with an incident id */
500: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorBody"];
};
};
/** @description The workspace is busy; retry after the header says */
503: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorBody"];
};
};
};
};
pre_label_batch: {
parameters: {
query?: never;
Expand Down Expand Up @@ -10821,5 +10972,6 @@ export interface KnownMembers {
ConnectionAction: "download_weights" | "check_integrity" | "update" | "delete";
JobAction: "start" | "complete";
ModelCapability: "point_suggest" | "text_detect";
PreLabelExclusionReason: "no_bbox_geometry" | "required_attribute";
SuggestParameter: "detail";
}
10 changes: 10 additions & 0 deletions frontend/ui-core/src/generated/checks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,15 @@ export const checkJobOut: Check<Schemas["JobOut"]> =
export const checkJobPage: Check<Schemas["JobPage"]> =
/*#__PURE__*/ object({ "items": [true, arrayOf(checkJobOut)], "total": [true, isInteger] } as const);

export const checkPreLabelExclusionReason: Check<Schemas["PreLabelExclusionReason"]> =
/*#__PURE__*/ openOneOf(["no_bbox_geometry", "required_attribute"] as const);

export const checkPreLabelExclusionOut: Check<Schemas["PreLabelExclusionOut"]> =
/*#__PURE__*/ object({ "name": [true, isString], "reasons": [true, arrayOf(checkPreLabelExclusionReason)] } as const);

export const checkPreLabelPlanOut: Check<Schemas["PreLabelPlanOut"]> =
/*#__PURE__*/ object({ "asked_classes": [true, arrayOf(isString)], "excluded_classes": [true, arrayOf(checkPreLabelExclusionOut)], "schema_version": [true, isInteger] } as const);

export const checkProjectOut: Check<Schemas["ProjectOut"]> =
/*#__PURE__*/ object({ "description": [true, either([isString, isNull] as const)], "id": [true, isString], "name": [true, isString] } as const);

Expand Down Expand Up @@ -412,6 +421,7 @@ export const checkListSchemaVersions = checkSchemaVersionPage;
export const checkListSources = checkSourcePage;
export const checkNextPendingAssets = checkAssetPage;
export const checkPreLabelBatch = checkBackgroundJobOut;
export const checkPreLabelPlan = checkPreLabelPlanOut;
export const checkPreviewSchemaChange = checkSchemaChangePreviewOut;
export const checkPromoteBatch = checkAssetPage;
export const checkPublishRelease = checkReleaseOut;
Expand Down
Loading
Loading