Add Logistic PCA (LPCA) analysis

[Jeebjean]

Summary

Adds Logistic PCA (LPCA) as a new SPRAS analysis method. LPCA is the binary-data counterpart to the existing PCA analysis. It is designed for binary matrices (0/1) like the edge-by-run matrix built from pathway reconstruction outputs, whereas standard PCA assumes continuous data.

Changes

New config block

analysis.lpca with five options:

• include: enable/disable LPCA (default false)
• k: number of principal components (default 2)
• m: logisticPCA tuning parameter, used when cv is false (default 6)
• cv: if true, determine m by cross-validation; if false, use fixed m (default false)
• transpose: if false, run LPCA on edges x runs matrix (one embedding per edge);
  if true, run on runs x edges matrix (one embedding per pathway run,
  mirrors the classic PCA analysis) (default false)

New Snakemake rule lpca_analysis

• Restricted to algorithms with multiple parameter combinations (same guard as per-algorithm PCA)
• Outputs per algorithm: {algorithm}-lpca-scores.csv, {algorithm}-lpca.png, {algorithm}-lpca-coordinates.txt

New module spras/analysis/lpca.py

• Builds the binary matrix via summarize_networks
• Optionally transposes the matrix
• Runs the logisticPCA container via run_container_and_log
• Generates a PC1 vs PC2 scatterplot and saves coordinates

New Docker wrapper docker-wrappers/lpca/

• Pinned rocker/r-base:4.4.2 Dockerfile installing logisticPCA and ggplot2
• run_lpca.R: runs LPCA and saves scores + deviance explained
• run_cv.R: cross-validation to find optimal m
• README documenting config options and Docker image publication

Unit tests test/analysis/test_lpca.py

• 3 tests: output file existence, score shape (edges x k), score shape (runs x k with transpose)

Docker timeout

• Increased from 60s to 600s in containers.py to handle larger binary matrices

Testing locally

Build the image and run the pipeline:

docker build -t reedcompbio/lpca:v1 docker-wrappers/lpca/

Enable analysis.lpca.include: true in your config and run as usual.
Tested end-to-end with OmicsIntegrator2 (216 combos) and MincostFlow (14 combos).

Before / at merge

The image must be published to reedcompbio/lpca:v1 on Docker Hub by a maintainer:

docker build -t reedcompbio/lpca:v1 docker-wrappers/lpca/
docker push reedcompbio/lpca:v1

cc @agitter @ntalluri

[read-the-docs-community]

Documentation build overview

📚 spras | 🛠️ Build #33784013 | 📁 Comparing b1905c0 against latest (1cb9d03)

  🔍 Preview build  

3 files changed

± genindex.html
± fordevs/spras.config.html
± fordevs/spras.html

[agitter]

Note that this will close #271 when merged.

I pushed the image to DockerHub: https://hub.docker.com/r/reedcompbio/lpca

We should add this to our Docker image building test: https://github.com/Reed-CompBio/spras/blob/main/.github/workflows/build-containers.yml

This comment should be modified to note the alternative LPCA

spras/spras/analysis/ml.py

Line 159 in 07707bf

# TODO: replace PCA https://github.com/Reed-CompBio/spras/issues/271

I didn't comment on every detail yet, but in general there are some small things we can to do make this more like the existing PCA.

[agitter]

Additional comments because I accidentally submitted the first batch before finishing.

[agitter]

Is this still needed? Was it a workaround for the memory issues?

This modifies Docker but not Singularity.

[Jeebjean]

The 600s timeout is still needed. Large matrices (e.g. RWR with ~20k edges) can take several minutes even with partial_decomp=TRUE. Singularity is unaffected.

[agitter]

A stronger test would be that the expected output is created when we use known inputs. These are okay first steps, but if there is time we would add the stronger test.

There is some degeneracy with these existing pathways though that cause the PCA tests to fail at random. It may be okay here because we don't do KDE testing.

Will we need to account for sign non-identifiability in LPCA?

[agitter]

Should this be a new rule or a conditional extension of ml_analysis and ml_analysis_aggregate_algo?

[Jeebjean]

Kept it as a separate rule to avoid touching existing ml_analysis logic. Happy to refactor it as a conditional extension and add cross-algorithm support if that's the preferred direction before merge.