From f695e1c3209669b29f2a1cf961d475a7638a7db1 Mon Sep 17 00:00:00 2001 From: Peter Jacobson Date: Fri, 12 Jun 2026 22:25:58 +1000 Subject: [PATCH] Refresh project docs against the current codebase for v1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CONTRIBUTING.md described gui/_legacy.py (~6,200 LoC) and cli/_legacy.py (~2,200 LoC) as holding the bulk of the implementation; the decomposition finished long ago and the relationship has inverted — gui/compat.py (47 lines) and cli/_legacy.py (124 lines) are thin re-export shims. The section now says so and forbids new code there. Also corrected en masse: - docs/cli.md: twelve shipped commands were missing from the table (facet-level, tv-denoise, edge, rotate*/flip*, periodicity, grains, diag-z). - docs/review_status.md: dated 2026-06-04; now records the 2026-06 adversarial review campaign (PRs #20-#36) and lists gui.md. - docs/core_derisk_plan.md: marked completed, kept as a design record. - docs/roi_manual_test_checklist.md: dropped the stale "Phase 4c / matplotlib/QScrollArea" framing; fixed the tool-shortcut line (G = polygon, P = point; the doc claimed P polygon / T point). - docs/notes/roi-display-notes.md: flagged as point-in-time notes. All architecture claims spot-verified against the code; relative links checked. createc_dat_reader.md, gui.md, and tests/README.md were already accurate. Co-Authored-By: Claude Fable 5 --- CONTRIBUTING.md | 28 ++++++++++++--------- docs/cli.md | 8 ++++++ docs/core_derisk_plan.md | 6 +++++ docs/notes/roi-display-notes.md | 4 +++ docs/review_status.md | 42 +++++++++++++++++++++++++++++-- docs/roi_manual_test_checklist.md | 10 +++++--- 6 files changed, 80 insertions(+), 18 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 084718f..69d3050 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -64,21 +64,25 @@ and what does **not**. Read these before adding new code: If a change crosses a boundary, prefer adding a small adapter in the caller over moving domain code into a foreign package. -## Where the implementation actually lives +## Compatibility shims -The package directory layout looks fully decomposed, but two large -files still contain the bulk of the GUI and CLI implementation: +The decomposition of the original monolithic GUI and CLI files is +complete: the directory layout is the real code layout, and every class +lives in its proper submodule (`gui/dialogs/`, `gui/viewer/`, +`gui/browse/`, `cli/commands/`, …). -- `probeflow/gui/_legacy.py` (~6,200 LoC) — main window, all dialogs, - panels, sidebars, the developer terminal. -- `probeflow/cli/_legacy.py` (~2,200 LoC) — every command parser and - dispatcher. +Two small shims remain only to keep the historical import surface +stable: -The other modules in `gui/` and `cli/` are mostly thin re-exports back -into these files. We are decomposing `_legacy.py` opportunistically: -when you touch a class for a feature, pull it out into its proper -submodule. Avoid standalone refactor sprints — they have no stopping -condition and break tests for too long. +- `probeflow/gui/compat.py` — re-exports consumed via + `gui/__init__.py` so `from probeflow.gui import X` keeps working. +- `probeflow/cli/_legacy.py` — re-exports every public CLI name into + the canonical `cli/parser.py`, `cli/processing_ops.py`, and + `cli/commands/*` modules. + +Do not add new code to either shim — new GUI or CLI code goes directly +in the appropriate submodule. When a re-export stops having external +users, it can simply be deleted. ## Commit style diff --git a/docs/cli.md b/docs/cli.md index 8e08ba1..1442e9d 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -21,16 +21,24 @@ Run `probeflow --help` for exact options. | `pipeline` | Apply ordered processing steps | | `prepare-png` | Write a PNG handoff with provenance sidecar | | `plane-bg` | Polynomial background subtraction | +| `facet-level` | Plane-level using only flat-terrace pixels (stepped surfaces) | | `align-rows` | Per-row offset correction | | `remove-bad-lines` | Detect and interpolate bad scan lines | | `smooth` | Gaussian smoothing | +| `tv-denoise` | Total-variation denoising (Huber-ROF / TV-L1) | +| `edge` | Edge detection (Laplacian / LoG / DoG) | +| `rotate`, `rotate-90`, `rotate-180`, `rotate-270` | Rotate a scan (arbitrary angle or lossless quarter turns) | +| `flip-h`, `flip-v` | Mirror a scan about the vertical / horizontal axis | | `fft` | Fourier-domain filtering | | `histogram` | Pixel-value histogram | | `fft-spectrum` | FFT magnitude spectrum | | `profile` | Line profile from endpoints or a named line ROI | +| `periodicity` | Find dominant spatial periodicities via power spectrum | | `autoclip` | Suggest display clip percentiles | | `particles`, `count`, `classify` | Feature analysis workflows | +| `grains` | Detect grains / islands by threshold and print statistics | | `lattice`, `unit-cell` | Lattice extraction and unit-cell averaging | +| `diag-z` | Diagnose Z-scale candidates for a Createc `.dat` file | | `spec-info`, `spec-plot`, `spec-overlay`, `spec-positions` | Spectroscopy utilities | Some feature/lattice commands require optional dependencies from the diff --git a/docs/core_derisk_plan.md b/docs/core_derisk_plan.md index c704967..bf78d4a 100644 --- a/docs/core_derisk_plan.md +++ b/docs/core_derisk_plan.md @@ -1,5 +1,11 @@ # Core de-risking plan +> **Status: completed (2026-06).** All phases are done or were deliberately +> re-assessed and closed with guard tests instead (Phases 2–3 below explain +> why). Kept as a design record: it documents *why* the `Scan` lazy imports +> and the `state.py` dispatcher look the way they do, so they aren't +> "tidied" into regressions later. + **Goal:** reduce the highest-risk structural brittleness in ProbeFlow's core — without changing behaviour — by working in small, independently-mergeable steps on short-lived branches. diff --git a/docs/notes/roi-display-notes.md b/docs/notes/roi-display-notes.md index c1ec62c..226df09 100644 --- a/docs/notes/roi-display-notes.md +++ b/docs/notes/roi-display-notes.md @@ -1,5 +1,9 @@ # ROI display & interaction — implementation notes +> Point-in-time implementation notes (2026-05); cited line numbers drift. +> The described architecture (hover/selection flow, `_ROI_HINTS`, +> `DisplayRangeController`, per-region composite) remains current. + Working notes for the per-region brightness/contrast + line-ROI click-fix work. Captures how the touched subsystems actually function so the in-app help/tooltip text (Part C) can be corrected against reality rather than assumption. diff --git a/docs/review_status.md b/docs/review_status.md index 973355d..b3898ff 100644 --- a/docs/review_status.md +++ b/docs/review_status.md @@ -1,6 +1,6 @@ # ProbeFlow Review Status -**Updated**: 2026-06-04 +**Updated**: 2026-06-12 This is the single, consolidated record of ProbeFlow's code-review history. The detailed per-angle review files (`docs/reviews/2026-05-27-*.md`) and the earlier @@ -10,7 +10,7 @@ findings are enacted. The conclusions and the remaining open items survive below ## Summary -Two review efforts have run on ProbeFlow: +Three review efforts have run on ProbeFlow: 1. **Staged review (2026-05, "Stage 1/2/3")** — scientific-workflow/physics pass, an architecture/maintainability pass with bounded refactor slices, and a @@ -19,6 +19,10 @@ Two review efforts have run on ProbeFlow: stability, image-processing pipeline, IO/instrument format, backend architecture, GUI architecture) producing **114 findings (S0=1, S1=36, S2=61, S3=16)**. +3. **Adversarial review campaign (2026-06-09 → 06-12, PRs #20–#36)** — targeted + seam-by-seam passes over the areas the earlier reviews could not exercise + (GUI timing, async loading, replay fidelity, parsers under corruption). See + the dedicated section below. **Status: essentially complete.** Every S0 and S1 finding, and every physics / numerical-correctness finding, has been enacted. What remains (listed below) is @@ -165,6 +169,38 @@ and a real import-churn/regression cost to change. Deliberately not pursued. `feature_points` / `feature_metadata` via `getattr` defaults — minor coupling, acceptable as-is. +## Adversarial review campaign (2026-06-09 → 06-12, PRs #20–#36) + +A third effort ran as short adversarial passes, each shipping its fixes with +regression tests before moving on. One PR per pass; the PR descriptions hold +the per-finding detail. + +- **GUI robustness (#20)** — QPixmap built off the GUI thread, worker-signal + lifetime (`SIGSEGV` class), thread-pool drain at exit. +- **Browse loading at scale (#21)** — network-drive freezes: async preview / + index workers, metadata peek budget, sliced card building, thumbnail + priority. +- **Seam review, two passes (#22–#26)** — scoped-filter replay ordering, + browse async/navigation seams, sidecar discovery fallback, corrupt-sidecar + visibility, quick-selection lifecycle, scale/shear overlay transforms, and + the PySide wrapper-recycling test flake (root-caused and fixed). +- **Physics reviews (#27, #29)** — spectroscopy: the qPlus setpoint gate + (Δf misread as amps), time-axis sanitisation, derivative units; FFT: + window-envelope compensation and odd-size soft-border centring. +- **User-feedback batch (#28, #30)** — mains custom streak pairs, notch-width + visualisation, background notch fill, streak/overlay decoupling. +- **Workflow-replay harness (#31)** — a permanent integration harness + asserting display == export == provenance-replay for representative + pipelines (plus a set-zero frame fix it caught). +- **Parser adversarial review (#32–#34)** — mutated-fixture corpus tests for + every reader; strict VERT metadata summaries (fast path now agrees with the + full parse), partial-load warnings surfaced in the viewer, Nanonis + single-column fix, and recognition of the normal Createc trailing appendix + (no more spurious warnings on healthy files). +- **Docs (#35–#36)** — GUI guide with offscreen-generated screenshots. + +Suite grew from ~2,260 to 2,449 tests across the campaign; main is green. + ## Deferred (not in code-review scope) - A true Python 3.11/3.12 test matrix (only the local interpreter was available). @@ -173,7 +209,9 @@ and a real import-churn/regression cost to change. Deliberately not pursued. ## Current user-facing / reference docs (kept) +- `docs/gui.md` - `docs/cli.md` - `docs/createc_dat_reader.md` - `docs/roi_manual_test_checklist.md` - `docs/notes/roi-display-notes.md` +- `docs/core_derisk_plan.md` (completed plan, kept as a design record) diff --git a/docs/roi_manual_test_checklist.md b/docs/roi_manual_test_checklist.md index 4dc9f48..73656a2 100644 --- a/docs/roi_manual_test_checklist.md +++ b/docs/roi_manual_test_checklist.md @@ -1,7 +1,9 @@ # ROI Manual Workflow Checklist -Phase 4c stabilisation checklist for the matplotlib/QScrollArea image viewer. -Run this on a representative STM image before moving on to DisplayLayer work. +Manual regression checklist for the image viewer's ROI tools. Run it on a +representative STM image after changes to the canvas, ROI items, or the ROI +Manager — it covers the interactions that automated GUI tests exercise least +(drag, hover, half-finished drawing, tool switching). ## Rectangle ROI @@ -45,9 +47,9 @@ Run this on a representative STM image before moving on to DisplayLayer work. ## Tool Behaviour -- Keyboard shortcuts switch tools: R rectangle, E ellipse, P polygon, F freehand, L line, T point, 1-9 ROI selection, I invert, Delete, Escape. +- Keyboard shortcuts switch tools: R rectangle, E ellipse, G polygon, F freehand, L line, P point, 1-9 ROI selection, I invert, Delete, Escape. - Escape cancels active drawing before closing the dialog/window. -- Pan mode left-drag on empty image pans the scroll area. +- Pan mode left-drag on empty image pans the view. - Pan mode drag on the active ROI moves the ROI. - Middle mouse always pans. - ROI drag-move does not conflict with panning.