Skip to content

Serve low-res volumes from a writable root; fix make_lowres for real dataset layout#73

Merged
aperson30 merged 1 commit into
mainfrom
perf/lowres-writable-root
Jul 16, 2026
Merged

Serve low-res volumes from a writable root; fix make_lowres for real dataset layout#73
aperson30 merged 1 commit into
mainfrom
perf/lowres-writable-root

Conversation

@aperson30

Copy link
Copy Markdown
Collaborator

Why

?res=low has been silently serving full-res because the low-res files never got generated. Two reasons make_lowres.py couldn't run on the server:

  1. Read-only mount — it wrote *_lowres.nii.gz next to the originals under PANTS_PATH, which is a read-only NFS mount. nib.save fails there (same class as the uint8-mask issue).
  2. Wrong input glob — it globbed PANTS_PATH/data/ImageTr|ImageTe/*/ct.nii.gz, which finds zero cases. The real layout is PANTS_PATH/image_only/<case>/ct.nii.gz + PANTS_PATH/mask_only/<case>/combined_labels.nii.gz.

(Server discovery: 9,901 cases, ~32 MB/CT; factor 2 ≈ 40–45 GB, factor 4 ≈ 5–6 GB; /home has 1.2 TB free.)

Changes

  • make_lowres.py: glob image_only/*/ct.nii.gz; write under --out-root (default /home/visitor/pants_lowres, or $PANTS_LOWRES_PATH), mirroring image_only/<case>/ + mask_only/<case>/; makedirs the output dirs; refresh docstring + conda interpreter.
  • api_blueprint.py: get-main-nifti / get-segmentations ?res=low now read from LOWRES_ROOT (same default/env) instead of the read-only mount.

Read and write paths use the identical convention, so they line up by construction.

Additive / safe

If the low-res files don't exist, both endpoints fall through to full-res exactly as today — a no-op until the batch runs. Nothing writes into the read-only image_only/ / mask_only/.

Deploy (after merge, on the server)

PYBIN=/home/visitor/.conda/envs/PanTS_backend/bin/python3.11
cd /home/visitor/PanTS-Viewer/flask-server
$PYBIN scripts/make_lowres.py --factor 2 --limit 5   # trial
nohup $PYBIN scripts/make_lowres.py --factor 2 > /tmp/make_lowres.log 2>&1 &   # full batch

No .env change needed (default matches). Restart the backend after the batch so ?res=low starts serving the new files.

Test plan

  • py_compile clean on both files
  • Trial run (--limit 5) writes to /home/visitor/pants_lowres/{image_only,mask_only}/<case>/…
  • After batch + restart: curl ".../api/get-main-nifti/<id>.nii.gz?res=low" returns the smaller file

…layout

The dataset mount is read-only, so make_lowres could never write *_lowres.nii.gz
next to the originals, and its input glob looked for data/ImageTr|ImageTe/ which
does not exist (real layout is image_only/<case>/ + mask_only/<case>/).

- make_lowres: glob image_only/*/ct.nii.gz; write output under --out-root
  (default /home/visitor/pants_lowres or $PANTS_LOWRES_PATH), mirroring
  image_only/<case>/ + mask_only/<case>/; makedirs; refresh docs/interpreter.
- api_blueprint: the get-main-nifti / get-segmentations ?res=low paths now read
  from LOWRES_ROOT (same default/env) instead of the read-only mount.

Fully additive: absent low-res files -> the API serves full res exactly as before.
@aperson30 aperson30 merged commit cea93cc into main Jul 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant