Skip to content

Go-public + PyPI readiness: packaging, onboarding docs, CI green (C-318/C-319/C-320) - #364

Merged
Polichinel merged 8 commits into
developmentfrom
feat/pypi-and-onboarding
Jul 27, 2026
Merged

Go-public + PyPI readiness: packaging, onboarding docs, CI green (C-318/C-319/C-320)#364
Polichinel merged 8 commits into
developmentfrom
feat/pypi-and-onboarding

Conversation

@Polichinel

Copy link
Copy Markdown
Contributor

What

Everything needed before making the repo public and publishing v1.9.0 to PyPI (first release), plus the first-time-user onboarding docs.

Pre-public security (Phase 0)

  • Redacted real SSH username and workstation hostname to placeholders (server runbooks + 2026-05-25 post-mortem)
  • Server hardened live: PasswordAuthentication no drop-in, verified key-only + fresh login (C-88 addendum)

PyPI packaging (Phase 1)

  • pyproject: authors, keywords, classifiers, project.urls (views-frames template; PEP 639)
  • publish_package.yml: Trusted Publishing (OIDC) workflow, first-publish-safe version guard
  • CI python 3.10 → 3.12 (requires-python is >=3.12)
  • docs/guides/publishing_to_pypi.md runbook (incl. TestPyPI rehearsal)
  • Verified: uv build + twine check pass; wheel = 9 code packages only; sdist = /src only — no data ships

Onboarding docs (Phase 2)

  • NEW docs/guides/model_consumer_quickstart.md — nothing → running a datafactory-backed views-models model
  • NEW docs/guides/README.md — guides index by task/audience
  • credential_setup.md: missing GDL_API_TOKEN section added; stale _resolve_storage_options path fixed (+ ADR-026)
  • README: PyPI install, prerequisites table, guides links, DEFAULT_REMOTE in examples
  • Docs convention: code examples use DEFAULT_REMOTE, never the bare IP

Test-suite integrity (found by this work's verification)

  • C-319 (Tier 2): in-process main() in test_consumer_provenance held the production pipeline lock for pytest lifetime → 42 suite errors on every full run since v1.8.1. Fixed via the cooperating-child env seam.
  • C-320 (Tier 3): four deploy-gate falsification tests failed in CI (shallow checkout, no GH_TOKEN) → CI red on every branch incl. the v1.8.1 release chain, unnoticed. Now skip-with-reason in CI, still enforce locally.
  • Register: C-318 [DEFER] (cleartext basic auth — conscious trade-off), C-319/C-320 registered-and-resolved, header counts + 8000-char guard trim

Verification

  • Full suite: 2289 passed, 0 failed, 0 errors (first fully green run since v1.8.1)
  • ruff clean; register machine-checks green
  • Wheel/sdist contents inspected — code only

🤖 Generated with Claude Code

Polichinel and others added 8 commits July 27, 2026 14:40
Pre-public hygiene: the admin account name and personal hostname were
committed in server runbooks and a post-mortem. Replaced with
<your-user> / <user>@<workstation> placeholders (matching the
convention already used in hetzner_deployment_guide.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…elease runbook

- pyproject: authors, keywords, classifiers, [project.urls] (mirrors the
  views-frames template; License:: classifier omitted per PEP 639)
- publish_package.yml: Trusted Publishing workflow copied from views-frames
  (release:published trigger, first-publish-safe version guard, uv build/publish)
- ci.yml: python-version 3.10 -> 3.12 (requires-python is >=3.12; CI never
  exercised a supported interpreter)
- docs/guides/publishing_to_pypi.md: release runbook incl. TestPyPI rehearsal

Verified locally: uv build succeeds, twine check PASSED, wheel ships only the
nine datafactory_* packages, sdist ships only /src + license/readme.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…edential gaps

- NEW model_consumer_quickstart.md: nothing -> running a datafactory-backed
  views-models model (verified against the actual dataloader path; the two
  ask-a-human items are explicit as Step 0)
- NEW docs/guides/README.md: index of all guides by task and audience
- credential_setup.md: GDL_API_TOKEN section (SHDI) was missing entirely;
  anonymous-sources note; stale _resolve_storage_options path fixed
  (dataset.py -> backends_zarr.py, also in ADR-026)
- README: PyPI install line, Prerequisites table (who needs which
  credentials), guides row in Where-to-Find-What, remote example now
  imports DEFAULT_REMOTE
- Consumer guides: code examples use DEFAULT_REMOTE instead of the bare
  IP (docs convention: the address appears only in netrc setup blocks);
  viewser_transition_guide install line -> PyPI floor pin

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ER] trade-off

Recorded alongside the go-public/PyPI decision so the known-temporary
status of plain-HTTP serving is on the record, not in our heads.
Tier 4; trigger: external consumers or non-courtesy-protected data;
cheap fix: domain + Caddy auto-HTTPS. Cross-refs C-97 (auth model vs
this entry's auth transport) and the server-hardening cluster.
Header: 317→318 IDs, 27→28 open, Tier 4 14→15. Register falsification
tests green (7/7).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… (C-320)

C-319 (Tier 2): test_consumer_provenance invoked generate_consumer_data.main()
in-process; since v1.8.1 that acquires the REAL /var/lock/views-pipeline.lock
and holds it for process lifetime — the pytest process became the holder and
all 42 later subprocess-writer tests were correctly refused by their own lock.
Fix: set VIEWS_PIPELINE_LOCK_HELD=1 (the existing cooperating-child seam)
around the in-process main(). Proven: poisoner+victim files green in one
pytest process.

C-320 (Tier 3): the four deploy-gate falsification tests (merge topology x3,
issue hygiene x1) assumed a full clone and authenticated gh — neither holds
in Actions, so CI was red on EVERY branch including the entire v1.8.1 release
chain, unnoticed. Fix: gates skip-with-reason where the environment cannot
answer (merge-base exit 128, gh unauthenticated) and still enforce locally.

Register: C-319/C-320 registered-and-resolved; header 318->320 IDs,
287->289 resolved; struck-through 103->105. Lesson recorded: release ritual
must confirm CI green on release commits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r guard

The Prior: chain grew past the search-window guard
(test_falsification_merge_readiness F1). Older segments were already
duplicated in git history — the line ends with 'Earlier history in git
log' by design. Kept: 2026-07-27 go-public update, 2026-07-22 ops
sitting, 2026-07-21 incident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…verified key-only

Pre-public sshd audit found PasswordAuthentication unset (= yes) on the
internet-open SSH port. Fixed live via /etc/ssh/sshd_config.d/99-key-only.conf,
verified with sshd -T and a fresh key login before closing the old session.
Go-public gate passed; IP-allowlist scope of C-88 stays [DEFER].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 401->PermissionError mapping only caught OSError, but aiohttp's
ClientResponseError (what a real no-netrc request raises) is not an
OSError — first-time users saw a raw client error instead of the
PermissionError + netrc hint the credential guide promises. Found
empirically by the TestPyPI clean-room rehearsal's no-credential probe;
the prior unit test simulated the failure as OSError, a fixture that
did not mirror the real exception type (C-315 lesson-family).

Non-OSError exceptions carrying 401/Unauthorized now map to the
documented PermissionError; everything else re-raises untouched.
Regression tests pin both branches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Polichinel
Polichinel merged commit 3f62bd3 into development Jul 27, 2026
4 checks passed
@Polichinel
Polichinel deleted the feat/pypi-and-onboarding branch July 27, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant