Skip to content

Modernise the dependency stack and pin it exactly - #179

Merged
FWao merged 8 commits into
mainfrom
fix/build-6
Aug 20, 2026
Merged

Modernise the dependency stack and pin it exactly#179
FWao merged 8 commits into
mainfrom
fix/build-6

Conversation

@FWao

@FWao FWao commented Aug 19, 2026

Copy link
Copy Markdown
Member
  • Python 3.13/3.14, PyTorch 2.11, current runtime dependencies
  • flash-attn / mamba-ssm / causal-conv1d from pre-built wheels at https://wheels.astral.sh/simple/cu130/ instead of source builds
  • transformers held at 4.57 (transformers 5 requires all_tied_weights_keys, absent in the TITAN/PRISM remote-code models), huggingface-hub<1.0 to match
  • Every requirement pinned to ==, enforced by scripts/check_dependency_config.py
  • CI: static dependency checks, CPU matrix, GPU extras without a driver, aarch64

Verified on aarch64/GB10 and x86_64/RTX A6000: uv sync --locked --extra gpu_all is wheels-only, torch 2.11.0+cu130 runs on GPU, full suite green including 21/21 extractor tests.

FWao added 6 commits August 6, 2026 18:38
Adds a stdlib-only checker plus pytest coverage for the invariants that keep
the dependency stack installable without a CUDA toolchain:

  * no direct .whl URLs (they hard-code Python ABI, arch and libc)
  * git dependencies use HTTPS and a full 40-character commit SHA
  * flash-attn/mamba-ssm/causal-conv1d local versions agree with the pinned
    torch minor and the CUDA channel of the Astral index
  * gpu_prebuilt stays a plain alias
  * uv is configured to refuse source builds of the three extensions

The checker reports 16 problems against the current configuration; the
following commits fix them. It reads only pyproject.toml, so CI can run it
before installing anything.
Replaces the hard-coded GitHub release wheel URLs with ordinary version
requirements served by https://wheels.astral.sh/simple/cu130/. The URLs
pinned a single Python ABI (cp313), libc and architecture, so every Python
or arch change needed a manual wheel hunt; uv now picks the right artifact.

The pinned versions keep their full local segment (`+cu.13.0.torch.2.10`) so
the CUDA and PyTorch build a wheel was compiled against stays explicit.

Also in this commit:

  * `no-build-package` makes uv refuse to build the three extensions from
    source, so an unsupported platform fails at resolution instead of running
    a CUDA-detecting setup.py.
  * `exclude-dependencies` drops the unconditional CUDA requirements the UNI,
    GigaPath and COBRA forks declare; xformers is re-added under STAMP's `uni`
    extra with the platforms it actually ships wheels for. This removes the
    duplicated `dependency-metadata` block for UNI and for flash-attn.
  * The CUDA-only extras are marked `sys_platform == 'linux'`.
  * COBRA switches from git+http to git+https, and CONCH is pinned to
    e1bd7e6ad11e0a6c4e74591f9a982374f239aa9a (the commit already resolved by
    the previous lockfile, so resolution is unchanged).
  * `gpu_prebuilt` becomes a deprecated alias of `gpu_all`.
  * Adds the missing `gpu` vs `gpu_all` conflict.

The checker gained a rule for PEP 508 URL requirements needing whitespace
before the `;` marker separator: uv accepts the tighter spelling but hatchling
rejects it, which broke `uv sync` while building STAMP's own metadata.
Widens the supported Python range to >=3.13,<3.15 and makes 3.14 the default
development version. Moves the ABI-pinned stack to torch 2.11.0 /
torchvision 0.26.0 and the matching Astral cu130 extension builds.

PyTorch 2.11, not 2.12: 2.12 wheels exist for torch, torchvision and
flash-attn, but the cu130 Astral index has no mamba-ssm or causal-conv1d
built against 2.12 (newest is `+cu.13.0.torch.2.11` for both). Since COBRA
needs both and gpu_all combines every extra in one environment, 2.11 is the
newest PyTorch that has pre-built wheels for all three extensions. The cu132
channel does not help: it carries flash-attn for 2.12 but publishes no
mamba-ssm or causal-conv1d at all.

Ordinary dependencies moved to their newest stable releases: jaxtyping
0.3.11, lightning 2.6.5, matplotlib 3.11.1, openslide-bin 4.0.1,
openslide-python 1.4.6, packaging 26.3, pillow 12.3.0, pydantic 2.13.4,
scikit-learn 1.9.0, scipy 1.18.0, tqdm 4.70.0, timm 1.0.28, opencv-python
5.0.0.93, transformers 5.14.1, huggingface-hub 1.26.1, plus ipykernel,
pyright, pytest and ruff in the dev group.

Two dependencies could not be upgraded:

  * numpy stays at ~=2.4.3. Under numpy 2.5 an unparametrised
    `numpy.typing.NDArray` exposes the `ScalarT` TypeVar, and beartype 0.22.9
    (latest) raises BeartypeDecorHintNonpepNumpyException for it. STAMP
    annotates with `npt.NDArray` throughout, so this breaks at import time.
  * pandas stays at ~=2.3.3 because lifelines 0.30.3 (latest) requires
    pandas>=2.1,<3.0.

Verified on Apple Silicon with Python 3.14.3: clean `uv sync --locked --extra
cpu --dev`, all 25 CPU-compatible extractors and encoders import, and the
test suite matches the pre-existing baseline. The `test_mil_predict_generic`
DataLoader worker segfaults reproduce identically on unmodified main and are
a pre-existing macOS test-isolation issue, not a regression.
Every workaround that existed to make the compiled extensions build is now
gone, each removed and re-tested individually against a CUDA-less Linux
container running `uv sync --locked --extra gpu_all`:

  * `dependency-metadata` for uni and for flash-attn (replaced by
    `exclude-dependencies`, done in the Astral migration commit)
  * `extra-build-dependencies` for flash-attn, gigapath and conch -- none of
    those three needs torch in its build environment; gigapath and conch use
    setuptools with static metadata, and flash-attn is no longer built at all

One workaround is retained, with the reason recorded in pyproject.toml:
`build-constraint-dependencies`. COBRA lists torch in `build-system.requires`,
so uv resolves a torch for COBRA's build environment separately from the one
being installed. A cold-cache build with the constraint removed resolves
torch 2.13.0 from PyPI and pulls the entire cuda-toolkit chain -- gigabytes of
download to build one wheel, against a different torch than STAMP runs. It can
go once COBRA drops torch from its build requirements.

CI is reorganised around what can actually break:

  * a dependency-configuration job running the static checker with no install
  * CPU tests on Linux x86_64 (3.13 and 3.14) and Apple Silicon (3.14)
  * an Apple Silicon job asserting no CUDA package is installed and no CUDA
    extension is built from source
  * GPU extras installed on a runner with no nvcc, no driver and an invalid
    CUDA_HOME, for both `gpu` and `gpu_all`, checking the installed local
    versions rather than trusting the lockfile
  * the same check on Linux aarch64
  * extractor tests split out, so model-weight downloads stay out of the
    dependency jobs

All jobs use `uv sync --locked` so CI cannot silently rewrite uv.lock, pin
astral-sh/setup-uv to the v9.0.0 commit rather than a mutable tag, and install
the uv version recorded in UV_VERSION.

Documentation now states the supported Python and platform matrix, that the
GPU extras are pre-built, that gpu_prebuilt is a deprecated alias, and why uv
is required rather than pip. The "build flash-attn from source" instructions
are replaced: source builds are refused by design. Also corrects
getting-started.md, which suggested `uv sync --all-extras` -- impossible, since
the cpu/gpu/gpu_all targets are declared mutually exclusive.
Verified on an NVIDIA machine (2x RTX A6000, driver 595.71.05, Linux x86_64,
Python 3.14): transformers 5.14.1 breaks the TITAN and PRISM slide encoders.

transformers 5 reads `self.all_tied_weights_keys` in
`modeling_utils._move_missing_keys_from_meta_to_device` while loading any
model. TITAN and PRISM are loaded with `trust_remote_code`, and their
Hub-hosted classes were written against transformers 4, so they do not define
that attribute:

    AttributeError: 'Titan' object has no attribute 'all_tied_weights_keys'

Both `tests/test_encoders.py::test_if_encoding_crashes[titan]` and `[prism]`
failed on transformers 5.14.1 and pass on 4.57.6. The fix belongs in the
MahmoodLab/TITAN and paige-ai/Prism repositories, not here.

These tests are skipped on macOS CPU, which is why the earlier CPU-only run
did not catch this. huggingface-hub follows the same pin because transformers
4.57 requires `huggingface-hub<1.0`.

Full suite on the GPU machine with this stack: 89 passed, 0 failed.
@FWao
FWao merged commit 11d3de6 into main Aug 20, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants