From 0f245591b80ec2b918559dea50482938d541de00 Mon Sep 17 00:00:00 2001 From: igerber Date: Sat, 6 Jun 2026 13:55:46 -0400 Subject: [PATCH] refactor: rename SyntheticDiDResults.placebo_effects -> variance_effects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The field held method-specific contents — placebo treatment effects ("placebo"), per-draw bootstrap ATT estimates ("bootstrap"), or leave-one-out estimates ("jackknife"); `variance_method` disambiguates — so `placebo_effects` was misleading. Rename it to `variance_effects` (aligns with the sibling `variance_method` field) and keep `placebo_effects` as a deprecated read-only @property alias removed in v4.0.0 (matches the lambda_reg/zeta deprecation convention; a deprecated property alias on a Results object is a new pattern for this codebase). - results.py: rename the dataclass field; add a `placebo_effects` @property that emits DeprecationWarning and returns variance_effects (read-only; it is a property not a field, so dataclasses.replace/asdict use variance_effects). Migrate the internal get_loo_effects_df() reads to self.variance_effects so normal use never routes through the alias. Add __setstate__ migrating legacy pickled state (`placebo_effects`, no `variance_effects`; <= 3.5.x) onto variance_effects so old pickles' draws survive unpickle via both names. - synthetic_did.py: rename the fit() local placebo_effects/_n -> variance_effects/_n and the Results constructor kwarg. The _placebo_variance_se* helper docstrings keep "placebo_effects" (they genuinely return placebo effects). - tests: migrate 16 attribute reads; add test_placebo_effects_deprecated_alias (warns + identity + read-only), test_variance_effects_access_emits_no_warning (internal reads don't trip the alias), and test_legacy_pickle_state_maps_ placebo_effects. Migration completeness verified with a -W error::DeprecationWarning sweep. - docs: autosummary RST (add variance_effects, keep placebo_effects for the deprecation release), REGISTRY, tutorial 03 (attribute access only; placebo prose kept), llms-full.txt (prose ptr + Attributes-table row). - CHANGELOG Changed + Deprecated; TODO rows removed. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 17 +++ TODO.md | 4 - diff_diff/guides/llms-full.txt | 3 +- diff_diff/results.py | 55 +++++++-- diff_diff/synthetic_did.py | 22 ++-- .../diff_diff.SyntheticDiDResults.rst | 1 + docs/methodology/REGISTRY.md | 4 +- docs/tutorials/03_synthetic_did.ipynb | 4 +- tests/test_estimators.py | 4 +- tests/test_methodology_sdid.py | 104 ++++++++++++++---- tests/test_survey_phase5.py | 4 +- 11 files changed, 168 insertions(+), 54 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b0b990933..d2c5f86f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - **`SyntheticControl` conformal inference (Chernozhukov, Wüthrich & Zhu 2021, *JASA* 116(536)).** Three opt-in `SyntheticControlResults` methods give valid p-values for the post-period effect trajectory and pointwise confidence intervals — what the in-space placebo / Firpo-Possebom test-inversion paths cannot. Unlike the Firpo path (which re-ranks the cross-unit placebo gaps), the conformal layer fits its **own** time-permutation-invariant constrained-LS synthetic-control proxy (CWZ §2.3 eqs 3–4 — simplex weights on raw outcomes over **all** periods under the null, no `V`-matrix, no intercept) and permutes residuals **over time** for the single treated unit (CWZ's exactness theory requires a time-symmetric proxy, which the headline ADH `V`-matrix fit is not). **`conformal_test(effect, q=1, scheme="moving_block", n_iid=10000, seed=None)`** computes the joint sharp-null permutation p-value (eqs 1–2) of `S_q(û) = ((1/√T*)·Σ_{t>T0}|û_t|^q)^{1/q}` (`q ∈ {1, 2, ∞}`); the proxy is fit once and only residuals are permuted (footnote 7). **`conformal_confidence_intervals(alpha=0.1, scheme="moving_block", bounds=None, n_grid=100, seed=None)`** returns pointwise per-period CIs by test inversion (Algorithm 1 — each period `t` uses `Z = (pre-periods, t)` with the other post-periods dropped, a clean `T*=1` test). **`conformal_average_effect(alpha=0.1, scheme="moving_block", bounds=None, n_grid=200, seed=None)`** returns a CI for the average post-period effect by collapsing the panel into non-overlapping `T*`-blocks and permuting the block residuals (Appendix A.1). Permutation schemes: `"moving_block"` (`Π_→` cyclic shifts, valid under serial dependence — the default) and `"iid"` (`Π_all`, sampled, finer p-values); both include the identity so the p-value floor is `1/|Π|` (no extra `+1`). Fail-closed handling for `<1` donor / unpickled result / non-finite panel / non-converged grid points (treated as indeterminate, not rejected) / grid-limited / empty / unbounded sets; a single donor and `T*≥T0` warn. Surfaced under `conformal_inference` / `get_conformal_grid_df()` and `DiagnosticReport`'s `estimator_native_diagnostics`; the analytical `se`/`t_stat`/`p_value`/`conf_int`/`is_significant` stay NaN throughout. Core in the new `diff_diff/conformal.py` (reuses the Frank-Wolfe simplex solver). *Deferred:* one-sided variants (§7), covariates folded into the proxy, and the AR/innovation-permutation path (Lemmas 5–7). +### Changed +- **`SyntheticDiDResults.placebo_effects` renamed to `variance_effects`.** The + array's contents are method-specific — placebo treatment effects + (`variance_method="placebo"`), per-draw bootstrap ATT estimates + (`"bootstrap"`), or leave-one-out estimates (`"jackknife"`) — so the old name + was misleading; the `variance_method` field disambiguates the contents. Read + `result.variance_effects` going forward. + +### Deprecated +- **`SyntheticDiDResults.placebo_effects`** is now a read-only alias for + `variance_effects` that emits a `DeprecationWarning` on access; it will be + removed in v4.0.0. The alias is a property, not a dataclass field, so it is + read-only (assignment raises `AttributeError`) and + `dataclasses.replace(result, placebo_effects=...)` no longer works / + `dataclasses.asdict(result)` now emits the `variance_effects` key — use + `variance_effects`. + ## [3.5.1] - 2026-06-02 ### Added diff --git a/TODO.md b/TODO.md index df635e941..6043288dd 100644 --- a/TODO.md +++ b/TODO.md @@ -175,7 +175,6 @@ Deferred items from PR reviews that were not addressed before merge. | R comparison tests spawn separate `Rscript` per test (slow CI) | `tests/test_methodology_twfe.py:294` | #139 | Low | | CS R helpers hard-code `xformla = ~ 1`; no covariate-adjusted R benchmark for IRLS path | `tests/test_methodology_callaway.py` | #202 | Low | | Validating the `.txt` AI guides (`diff_diff/guides/llms-full.txt`, `llms-practitioner.txt`) as executable snippets is **not low-lift** (re-scoped 2026-06-01): of their ~112 fenced Python blocks only ~20% are standalone-runnable — the rest are API-signature references (`Foo(param: type = default)` pseudo-signatures that are `SyntaxError` by design), context fragments (e.g. `results.att` on an undefined `results`), or dataset-shape-specific blocks. The guides are reference documentation, not runnable examples; a real implementation needs signature-block detection + a context/data skip-allowlist + per-snippet fixtures (multi-round curation), unlike the curated `.rst` files the existing smoke test covers. | `tests/test_doc_snippets.py` | #239 | Low | -| SyntheticDiD: rename internal `placebo_effects` variable to `variance_effects` (or `resampled_effects`). Misleading name across the placebo/bootstrap/jackknife dispatch paths — holds three different contents depending on variance method. Low-risk refactor; user-facing field rename should preserve `placebo_effects` as a deprecated alias for one release. | `synthetic_did.py`, `results.py` | follow-up | Medium | | `TestWorkflowDoesNotExecutePRHeadCode` (CodeQL #14 dismissal guard) does not model: `bash