Skip to content

#805: Add node p-value display to plot_causal_tree - #976

Merged
jeongyoonlee merged 2 commits into
uber:masterfrom
genrichez:feat/plot-causal-tree-pvalue
Jul 27, 2026
Merged

#805: Add node p-value display to plot_causal_tree#976
jeongyoonlee merged 2 commits into
uber:masterfrom
genrichez:feat/plot-causal-tree-pvalue

Conversation

@genrichez

Copy link
Copy Markdown
Contributor

Proposed changes

Add the ability to display treatment effect p-values at each node when plotting a causal tree via plot_causal_tree. This addresses #805.

When node_pvalues=True is passed to CausalTreeRegressor during fit, a Welch's t-test is computed at each node comparing treatment vs control outcomes. The resulting p-values can then be rendered in the plot by passing pvalue=True to plot_causal_tree.

Usage:

model = CausalTreeRegressor(control_name=0, groups_cnt=True, node_pvalues=True)
model.fit(X=X, treatment=treatment, y=y)
plot_causal_tree(model, pvalue=True)

Each node will display p_value(group) = X.XXXX for each treatment group. Nodes with fewer than 2 samples in either group show N/A.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in downstream modules

Further comments

Implementation follows the same pattern as the existing groups_cnt feature:

  • node_pvalues=True in the constructor enables computation during fit()
  • _compute_node_pvalues() traverses the tree via decision_path (same approach as _count_groups_distribution)
  • Uses Welch's t-test (scipy.stats.ttest_ind, equal_var=False) which is standard for unequal group sizes
  • Multi-treatment support: computes a separate p-value for each treatment group vs control
  • Gracefully handles nodes with insufficient samples (returns None)

This PR was co-authored with AI assistance.

@CLAassistant

CLAassistant commented Jul 27, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@jeongyoonlee jeongyoonlee left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node p-values are naive in-sample within-node t-tests — the partition is learned from the same data and splits maximize effect separation, so they're anti-conservative (no honesty/selective-inference or multiple-testing correction) and are raw diff-in-means (valid only under in-node unconfoundedness). Fine as the diagnostic #805 asks for, but please add a docstring note on node_pvalues / plot_causal_tree(pvalue=...) that these are descriptive, not valid post-selection inference.

Minor: a zero-variance node makes ttest_ind return nan, rendered as p_value(g) = nan — worth mapping to N/A like the insufficient-samples branch.

@genrichez

Copy link
Copy Markdown
Contributor Author

The node p-values are naive in-sample within-node t-tests — the partition is learned from the same data and splits maximize effect separation, so they're anti-conservative (no honesty/selective-inference or multiple-testing correction) and are raw diff-in-means (valid only under in-node unconfoundedness). Fine as the diagnostic #805 asks for, but please add a docstring note on node_pvalues / plot_causal_tree(pvalue=...) that these are descriptive, not valid post-selection inference.

Minor: a zero-variance node makes ttest_ind return nan, rendered as p_value(g) = nan — worth mapping to N/A like the insufficient-samples branch.

Thanks for the thorough review! I've addressed these.
I've added the cautionary docstring notes for node_pvalues in both CausalTreeRegressor and plot_causal_tree, clarifying that these are descriptive p-values and not for post-selection inference ,also implemented the NaN to N/A mapping for p-values in the plot, and added a regression test to ensure this behavior is consistent.

@jeongyoonlee
jeongyoonlee merged commit f4526ac into uber:master Jul 27, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants