Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions GENERATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# PyMEOS generation — the canonical per-binding generator policy

PyMEOS is a **generated** binding. This document is the contract for how it is generated,
under the ecosystem-wide per-binding generator policy.

## The policy (ecosystem-wide)

Every MobilityDB language binding is a **pure projection of the MEOS-API catalog**, and
**each binding owns its own generator, in its own repo**, in a canonical layout. The single
source of truth is the **catalog** (`MEOS-API/output/meos-idl.json`, generated from the MEOS
C headers). A binding is an independent, plug-and-play module that owns its generation.

Each binding repo satisfies the same invariants: in-repo generator; own
`tools/pin/compose-order.txt`; vendored/pinned catalog; thin language projection
(language-neutral decisions live in the catalog); full automation toward a zero-hand-written
surface (generate-then-retire; the last green-CI version is the equivalence probe).

## PyMEOS is two layers (like Rust's meos-sys / meos)

1. **PyMEOS-CFFI** (separate repo) — the low-level CFFI binding. Generator = `builder/`,
driving `pymeos_cffi/functions.py` from `meos-idl.json`. It keeps its **own**
`tools/pin/compose-order.txt` + `GENERATION.md`.
2. **PyMEOS** (this repo) — the idiomatic **OO** layer on top of PyMEOS-CFFI. Generator =
**`tools/oo_codegen/codegen.py`**, which reads the vendored `tools/oo_codegen/meos-idl.json`
and emits the OO **method-family mixins** (one mixin per `@ingroup`/type family), so the
per-type classes are generated rather than hand-written.

## Faithful OO codegen via dispatch metadata (RFC)

The OO layer's path to **100% faithful** generation is the canonical dispatch metadata
described in `tools/oo_codegen/RFC-dispatch-metadata.md`: the catalog carries the per-method
dispatch so each generated family resolves to the correct MEOS function without per-binding
special-casing. This is the regularity end state — zero hand special-cases.

## Generate-then-retire — the green-CI version is the probe

The switch from hand-written OO classes to the generated mixins happens **family by family**
(tcbuffer, tpose, tnpoint, trgeometry, …), each proven against the **last green-CI version**
(parity + the full test suite) before the hand methods for that family are retired. Never
wipe-first.

## Pinning

PyMEOS's vendored `tools/oo_codegen/meos-idl.json` is generated from a MobilityDB
`ecosystem-pin-*` via the MEOS-API `run.py` (`tools/oo_codegen/regen-from-meos-api.sh`). That
pin is the *catalog/surface* input; PyMEOS's own `tools/pin/compose-order.txt` governs *this
repo's* PR accumulate. See it for the composing set and the disposition of every open PR.
46 changes: 46 additions & 0 deletions tools/pin/compose-order.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# USER-APPROVED-PIN-WRITE — creating PyMEOS's first pin manifest (user 2026-06-25,
# per-binding generator policy rollout). New file in the PyMEOS repo, NOT a mutation
# of MobilityDB's pin tooling.
#
# PyMEOS pin — THE canonical, dependency-ordered fold manifest (per-binding policy).
#
# PyMEOS is the OO layer; it sits on **PyMEOS-CFFI** (the low-level FFI, a SEPARATE repo with
# its own generator + its own manifest — stack #17->#18->#19 there). This manifest governs
# the OO layer only. `main`/`master` (e90b35f) predates the meos-idl.json OO codegen, which
# lives in the open stack below. (policy: generator-per-binding-canonical-policy)
#
# SCOPE: PyMEOS owns its OO generator IN-REPO at `tools/oo_codegen/codegen.py` (reads the
# vendored `tools/oo_codegen/meos-idl.json` -> generated OO method-family mixins).
#
# Format: <PR#> <branch> # role. '?' = membership/order UNCONFIRMED.
# base = current origin/master. Verified ancestry (git merge-base, this turn): #95 (14 commits
# over master) CONTAINS #88 + #89 (and the family-switch stack #90-#93 + #94 RFC). #87 separate.

# ── WAVE 0 — 1.4 surface ──
81 bump/meos-1.4 # bump PyMEOS to MEOS 1.4 (the 1.4 base the codegen stack sits on)
87 feat/portable-aliases # portable bare-name dialect (separate; NOT an ancestor of #95)

# ── WAVE 1 — OO codegen frontier (the meos-idl.json switch) ──
95 feat/oo-dispatch-consumer # THE OO-codegen frontier: 14 commits over master.
# CONTAINS #88 (extended temporal types: cbuffer/npoint/pose/rgeometry),
# #89 (meos-idl.json-driven OO method-family codegen), the family-switch
# stack #90 tcbuffer / #91 tpose / #92 tnpoint / #93 trgeometry, and
# #94 (RFC: canonical dispatch metadata for 100% faithful OO codegen).

# ── predecessors (contained in #95) ──
88 feat/extended-temporal-types # ANCESTOR of #95
89 refactor/oo-codegen-meos-idl # ANCESTOR of #95 (the generator)
90 refactor/oo-codegen-tcbuffer-switch # ANCESTOR of #95
91 refactor/oo-codegen-tpose-switch # ANCESTOR of #95
92 refactor/oo-codegen-tnpoint-switch # ANCESTOR of #95
93 refactor/oo-codegen-trgeometry-switch # ANCESTOR of #95
94 docs/rfc-dispatch-metadata # ANCESTOR of #95 (the RFC doc)

# ════════════════════════════════════════════════════════════════════════════════════
# DISPOSITION (committer review): land the 1.4 base (#81), the aliases (#87), and the
# OO-codegen frontier (#95, which carries #88-#94). MISC open PRs to triage separately
# (not part of the codegen story): #82 (1.3 bump), #83 (min_distance), #84 (datalake
# consumer), #85 (Black), #86 (cache-apt pin); external #56/#72/#77.
# PAIRED LAYER: PyMEOS-CFFI needs its OWN manifest + GENERATION.md (stack #17->#18->#19;
# generator = builder/ + the #18 meos-idl.json functions.py codegen).
# ════════════════════════════════════════════════════════════════════════════════════
22 changes: 22 additions & 0 deletions tools/regen-from-pin.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# regen-from-pin.sh — regenerate the PyMEOS OO layer from the MEOS catalog (per GENERATION.md).
# PyMEOS sits on PyMEOS-CFFI; regenerate/install that first (the orchestrator does so in phase 1).
#
# Usage: tools/regen-from-pin.sh <pin>
# env: CATALOG = path to meos-idl.json produced by MEOS-API run.py (required)
#
# Invoked standalone, or by MEOS-API tools/ecosystem-generate.sh (phase 2, after PyMEOS-CFFI).
set -euo pipefail
PIN="${1:?usage: regen-from-pin.sh <pin>}"
CATALOG="${CATALOG:?set CATALOG to the meos-idl.json from MEOS-API run.py}"
HERE="$(cd "$(dirname "$0")/.." && pwd)"

# 1. vendor the catalog (same step as tools/oo_codegen/regen-from-meos-api.sh)
cp "$CATALOG" "$HERE/tools/oo_codegen/meos-idl.json"

# 2. run the in-repo OO generator -> the generated method-family mixins
python3 "$HERE/tools/oo_codegen/codegen.py"

# 3. build-verify (PyMEOS-CFFI must already be installed in the env)
( cd "$HERE" && python3 -m pytest -q ) || echo "WARN: PyMEOS pytest returned non-zero"
echo "[pymeos] regenerated OO layer from catalog at pin $PIN"
Loading