[DOCS] Consolidate the Python docs into SCHEMA_GUIDE.md - #684
Draft
Seth Fitzsimmons (sethfitz) wants to merge 7 commits into
Draft
[DOCS] Consolidate the Python docs into SCHEMA_GUIDE.md#684Seth Fitzsimmons (sethfitz) wants to merge 7 commits into
Seth Fitzsimmons (sethfitz) wants to merge 7 commits into
Conversation
Discovery emitted `feature` and `overture:theme=*` but never the plain `overture` tag, so the filter the CLI's own help suggested matched nothing. Add an `overture_provider` in overture-schema-common that attaches the tag when any concrete arm of an entry point subclasses `OvertureFeature`, and reserve the tag to that package. "Is this built on Overture's feature model?" is now answerable by tag, so a consumer asking it needs neither a dependency on this package nor an `issubclass` call. The tag is not a claim that the type belongs to the Overture schema. Any package can subclass `OvertureFeature` and register an entry point, and this provider tags it like any other; reserving the tag governs who may emit it, not which models receive it. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
The command docstrings were raw strings, so Click's `\b` no-rewrap marker stayed two literal characters and every example block collapsed into one paragraph. Drop the `r` prefix; pydocstyle's D301 pushes the other way and is a false positive for Click docstrings, so it is waived with the reason recorded once at the module level. The "official Overture types only" examples suggested `--tag overture --tag feature`, wrong twice over: the plain `overture` tag did not exist, and `--tag` is OR, so pairing it with `feature` widens the scope instead of narrowing it. Cite `--tag overture` alone, reworded -- the tag says a type is built on Overture's feature model, not that it is official. Added an example showing what the OR is for, and tag examples to `list-types`, which had none. The `--tag` syntax note illustrated namespaced tags with `overture:approved`, which no provider can register. No shipped model emits a bare namespaced tag, so the note describes the form instead. `test_help_cites_only_tags_that_exist` covers the whole surface: it splits rendered help per option and checks both citation forms -- an argument (`--tag X`) and an illustration (`(e.g. X)`) -- against the set that option accepts, since `--group-by` takes the key half of a `key=value` tag and `--tag` does not. Judged against a global union instead, a phantom passes in whichever option it was not written for. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
The listing fell back to `str(model)` for anything without `__name__`, so `Segment` printed its whole `typing.Annotated[...]` type expression -- several hundred characters of discriminator internals in place of a name. Read the name off the entry point instead. Every registered model has one, class or union alias alike, and it is the name the rest of the toolchain already uses. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Fixes the inaccuracies catalogued in #604 and #668. `packages/overture-schema/README.md` documented `from overture.schema import Building, parse, discover_models, json_schema`. None of it resolves: `overture.schema` is a namespace root shipping only `py.typed` since #622 moved the validation API out of it, and `parse()` exists nowhere. Rewritten against the real surface -- models from their theme packages, `validate`/`validate_json` from `overture.schema.validation`, discovery and `json_schema` from `overture.schema.system`. Three READMEs claimed `model_validate()` accepts GeoJSON. It does not, and the framing above it was also wrong: Overture publishes one shape, flat and tabular. The models additionally read and write GeoJSON so the schema works with tools that expect features rather than rows, and because that is the representation the generated JSON Schema describes. The two modes are not interchangeable, and the error a GeoJSON dict produces ("theme Field required") does not point at the cause. Round-trip examples now pass `by_alias=True`, without which `class_` is emitted and the output will not re-validate. The codegen README described `analyze_type()` returning a `TypeInfo` with `.kind`/`.base_type` and imported `TypeKind`. Neither exists; it returns a `(FieldShape, bool, str | None)` tuple. Replaced with a worked example whose output is transcribed from a run, and corrected the layer diagram and the package credited with `discover_models`. The pyspark README's S3 examples named release 2026-06-17.0, which the bucket no longer holds. Bumped, with a note on reading the current identifier from the STAC catalog, since a hardcoded release goes stale by design. PYDANTIC_GUIDE.md prescribed a `models.py` / `enums.py` / `types.py` split inside a per-type subdirectory (#604). No theme package has such a file. Rewritten against the settled layout: one module per feature type at the theme root, `_common.py` for what a theme shares, a subpackage only for a type large enough to split. The container-to-mixin example now uses the real `Named` / `Appearance` mixins `Building` inherits. It also named two enum members that do not exist -- `Relationship` has `COMPOSITION`, `AGGREGATION`, `HIERARCHY`, `ASSOCIATION`, and `CONNECTS_TO`/`BELONGS_TO` are *roles*. A third instance was in overture-schema-system's README. And its `$defs` migration example placed an `Address` with `freeform`/`locality` in the addresses theme, claiming `Building` carries one; that model is `places.Address`, and `Place` is what holds it. Removed the Reference section's five "Complete Templates": they prescribed the dead file-name convention, and their content survives in Quick Start, Quick Reference, and the Relationship Patterns section. Python embedded in Markdown is now formatted with `ruff format`, so documented snippets match the style of the code they describe. That needs ruff 0.16, so the dependency floor moves from 0.13. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Documentation drifts silently: a module moves, a helper is renamed, and the README keeps confidently describing the old surface. #489, #604 and #668 are all instances. This makes the class fail the suite instead of accumulating. `tests/test_documented_imports.py` parses every fenced Python block in every tracked Markdown file and checks two things: that each `overture.*` import resolves, and that each `Enum.MEMBER` named exists on the enum. The second is not redundant -- `from ... import Relationship` resolves whether or not `Relationship.CONNECTS_TO` does. Executing the blocks outright is not viable: most fail on undefined names, because a documentation fragment legitimately omits its imports and uses placeholder names the prose supplies. Imports and enum members are the granularity that separates a defect from a fragment. Block extraction and import parsing are separate, unit-tested functions. The corpus cannot test either: a form the matcher misses simply yields nothing, and a sweep that collects less still passes -- so the fixtures are inline rather than drawn from repo content. Blocks indented inside list items are dedented before parsing, since an `IndentationError` is a `SyntaxError` and would otherwise be filed as "expected unparseable". Blocks that are deliberately not valid Python are pinned by a digest of their body rather than a count, which would stay put when one breaks as another is fixed; and an excused block may not contain an `overture` import, so the waiver cannot swallow the API citation it excuses. The tests live outside `packages/` because their subject is the repo's Markdown, including root-level files belonging to no package, and they import across every package -- `overture.schema.codegen` among them, which no single distribution depends on. `tests/README.md` records the admission rule. `make check` reaches the tree through ruff, `ruff format`, mypy and pytest; a bare `pytest packages/` does not. `lint-only` also checks Python embedded in Markdown, with `make format` as the counterpart. At the old ruff floor that check silently finds no files, and CI resolves one job at `lowest-direct`. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Signed-off-by: Dana Bauer <dana.bauer@gmail.com> Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
The consolidation was written against main before the doc fixes landed, so it brought the stale content forward and catalogued the rest. Ported from the corrected PYDANTIC_GUIDE.md, extracted from that file rather than retyped: the module-organization section, the import-organization example, and the container-to-mixin example built on the real `Named` / `Appearance` mixins. Section 3.2 is reframed too -- Overture publishes one shape, and GeoJSON is the representation the models support for compatibility and that the generated JSON Schema describes; the renamed heading's anchor is chased through GLOSSARY.md and two in-guide links. "Docs in the repo that are currently wrong" listed eight items; seven are fixed, so only `pip install` survives, pointing at the section that explains it. GLOSSARY.md's Tag entry said there is no plain `overture` tag, which was true when written and is what prompted adding one; it now reads eight tags, with a note on what the tag does and does not assert. The transcribed `list-types` output was regenerated by running the command. The imported guide also exposed two holes in the drift detector. A `>>>` REPL block failed to parse and dropped out of the sweep whole, taking its imports with it. And the guide deliberately shows `from overture.schema import Building` as a counter-example, which the detector read as a defect; a line carrying the counter-example glyph is exempt. Both have unit fixtures and a mutation that fails them. With those handled and the validation README's block fixed, no block in the repo is excused. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Seth Fitzsimmons (sethfitz)
added
the
change type - documentation - member 📝
9 tasks
🗺️ Schema reference docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
Seth Fitzsimmons (sethfitz)
marked this pull request as draft
August 19, 2026 20:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces #672. Same consolidation —
PYDANTIC_GUIDE.mdandREADME.pydantic.mdbecome a singleSCHEMA_GUIDE.md, linked fromREADME.mdandCONTRIBUTING.md, andGLOSSARY.mdgrows from 6 to 22 terms — rebased onto the documentation fixes in #683, with the corrections carried into the new guide. Dana Bauer (@danabauer)'s commit is preserved with its authorship; the second commit is the port.#672 can be closed in favour of this; it does not rebase cleanly, because the files it deletes are the files #683 edits.
Why this replaces it rather than stacking on it
#672 was written against
mainbefore the doc fixes landed, so it did two things this branch undoes:It carried the stale content forward. [DOCS] PYDANTIC_GUIDE.md documents a package layout the repo no longer uses #604's
models.py/enums.py/types.pyfile-type rules, the per-type-subdirectory prescription, and# In buildings/models.pyall survive verbatim intoSCHEMA_GUIDE.md. Those sections are now taken from the correctedPYDANTIC_GUIDE.md— extracted from the file programmatically rather than retyped, so the two cannot disagree.It catalogued the defects instead of fixing them. The guide gained a "Docs in the repo that are currently wrong" section listing eight items. Seven are fixed in [BUG] Correct the Python package documentation #683. Only
pip installsurvives, and it now points at the section explaining why. A standing list of known-wrong docs is a workaround for not having a check; [BUG] Correct the Python package documentation #683 adds the check.Other corrections
The transcribed
overture-schema list-typesoutput (two copies) and the--group-byoutput were regenerated by running the commands, so they show theoverturetag that #683 adds. TheModelKeyexamples were showingBuildingModel/PlaceModel, which are not the names of anything.GLOSSARY.md's Tag entry said "Seven tags exist today ... There is no plainoverturetag, despite what some help text suggests." Accurate when written, and the observation is what prompted #683 to add the tag rather than delete the help that referenced it. Now eight, with a note on what the tag does and does not assert — see the open question in #683, which is the place to weigh in on it.What importing the guide exposed
SCHEMA_GUIDE.mdbroke the drift detector twice on arrival, which is a fair argument for having it:>>>REPL block failed to parse and dropped out of the sweep whole, taking its imports with it. Transcripts are unwrapped now.from overture.schema import Building # ✗ ImportErroras a counter-example, and the detector read it as a defect. A line carrying the counter-example glyph is exempt.Both behaviours have unit fixtures and a mutation that fails them.
Review notes
overture.*import resolves, every internal link resolves. Two of those are now enforced bytests/test_documented_imports.pyon every run rather than checked once.SCHEMA_CONVENTIONS.mdstaleness flags inREADME.mdwere spot-checked and are accurate: it does describe JSON Schema as the definition mechanism (line 3) and does spellsubType(line 37).CONTRIBUTING.mdstill does not mention the DCO sign-off requirement. Carried over unaddressed, as in [DOCS] test, update, revise, and consolidate schema guides #672.Closes #685