Skip to content

AnwarDebes/ConfTM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ConfTM

Release Python License: MIT Tests Paper

The ninth member of the Tsetlin Machine family (alongside TM, GTM, HTM, HGTM, THGTM, RGTM, CCTM, DiTM).


ConfTM is a thin, model-agnostic wrapper around any TM-family classifier that produces:

  • Calibrated prediction sets with finite-sample marginal coverage $\Pr(y \in \hat C_\alpha(x)) \ge 1 - \alpha$ under exchangeability.
  • Mondrian (class-conditional) calibration: per-class coverage.
  • Adaptive online conformal (ACI): drift-resilient streaming calibration.
  • Coverage-guaranteed counterfactual recourse: bridges conformal prediction with CCTM-style recourse.

Pure-NumPy reference implementation; pairs naturally with every other TM in the family.

The paper is at paper/conftm.pdf.

Quick start

pip install -e .
pytest -q tests/         # 11 unit tests, ~18s
make reproduce           # all 4 experiments, ~3min on CPU
make paper               # build paper/conftm.pdf

What's in the box

Path Purpose
conftm/ta.py Canonical Tsetlin Automaton bank
conftm/tm.py Binary + multi-class Tsetlin Machine
conftm/scores.py Three TM-native nonconformity scores
conftm/conformal.py ConfTM: split-conformal wrapper
conftm/mondrian.py MondrianConfTM: class-conditional calibration
conftm/adaptive.py AdaptiveConfTM: online ACI under drift
conftm/recourse.py Coverage-guaranteed recourse
conftm/data.py Bundled synthetic medical-risk dataset
tests/ 11 unit tests, all green
experiments/coverage.py Marginal coverage at $\alpha \in {0.05, 0.10, 0.20, 0.30}$
experiments/efficiency.py ConfTM vs naive softmax-cutoff baseline
experiments/drift.py Static vs adaptive under covariate shift
experiments/cov_recourse.py Coverage-guaranteed recourse vs naive
scripts/make_figures.py Build every figure from results/*.json
paper/conftm.tex, references.bib Paper source
paper/figures/*.pdf Real figures from real numbers

Headline results

Experiment Result
Marginal coverage holds across $\alpha$ ✓ (every variant within tolerance)
Mondrian per-class coverage @ $\alpha=0.10$ $[0.95, 0.90, 0.97, 0.90]$ for 4 classes
Adaptive coverage under drift $0.900 \pm 0.0003$ vs static $0.836 \pm 0.014$ (target 0.9)
Cov-guaranteed recourse robustness @ $\alpha=0.10$ $0.913 \pm 0.012$ flip rate under feature noise
Receipts verified (handled in CCTM/THGTM; composes with ConfTM)

Honest limitations

  • Argmax vs label coverage. The conformal guarantee is on the label set, not the model's argmax. At very tight $\alpha$ this distinction can show up; we report it transparently (paper §4.4 and §6).
  • Exchangeability assumption. Static conformal needs calibration and test data exchangeable; adversarial shift breaks it. The adaptive variant handles distributional drift but not adversarial.
  • Synthetic data. All experiments use a bundled synthetic medical- risk dataset. Real-world evaluation (Adult-Income, MIMIC, COMPAS) is future work.
  • Class-imbalanced calibration. Mondrian needs ≥10-20 calibration samples per class for stable per-class quantiles.

License

MIT.

Citing

@misc{debes2026conftm,
  title  = {ConfTM: A Conformal Tsetlin Machine with Coverage-Guaranteed
            Counterfactual Recourse},
  author = {Debes, Anwar},
  year   = {2026},
  note   = {Reference implementation v0.1, May 2026}
}

About

Conformal prediction for Tsetlin Machines: wrap any TM and get prediction sets with finite-sample coverage guarantees. Split, Mondrian, and drift-adaptive calibration, plus recourse that stays valid under noise.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors