Skip to content

Add observational ITE benchmark DGP + all-CI demo notebook#23

Merged
oduerr merged 3 commits into
mainfrom
feat/ite-observational-dgp
Jun 26, 2026
Merged

Add observational ITE benchmark DGP + all-CI demo notebook#23
oduerr merged 3 commits into
mainfrom
feat/ite-observational-dgp

Conversation

@oduerr

@oduerr oduerr commented Jun 26, 2026

Copy link
Copy Markdown
Member

Adds a confounded-treatment individual-treatment-effect (ITE) benchmark to
the simulation portfolio (like VACA/CAREFL) plus a demo notebook training an
all-CI TRAM-DAG that recovers per-individual effects.

DGP ported from code/ITE_observational_simulation.R in
mikekr97/MA_Mike (the master's-thesis repo).

The DGP — simulations.ITEObservational (ite-observational)

A 7-variable mediation SCM, topological order X1 X2 X3 Tr X5 X6 Y:

X1, X2, X3 ~ N(0, Σ)            correlated, ρ=0.1
Tr ~ Bernoulli(sigmoid(0.5 - 0.5·X1 + 0.3·X2))      # treatment confounded by X1,X2
X5 :  2.5·X5 = logit(U5) - 0.8·Tr                    # Colr mediator
X6 :  4·X6   = logit(U6) + 0.5·X5                    # Colr mediator
Y  :  h_y(Y) = logit(U7) - [1.5·Tr + X·β + (-0.9·X2 + 0.7·X3)·Tr]

with β = (-0.5, 0.5, 0.2, -0.6, 0.4) on (X1,X2,X3,X5,X6) and a nonlinear
baseline
h_y(y) = tan(y/2)/0.2 on [-2,2] (slope-matched linear tails, so
h_y⁻¹(z) = 2·atan(0.2·z) in the core). All latents are logistic → the DGP sits
inside the flow's family; the shapes and the (X2,X3)·Tr interaction are what
must be learned.

Treatment flows through a direct path and a mediated path X5 → X6 → Y, and
the interaction makes effects heterogeneous. Four scenarios toggle
main/interaction effects (scenario=1..4); scenario 1 (main + interaction)
is the frozen/featured dataset. The generator exposes per-individual ground
truth — ITE_true (observed latent) and ITE_median (median latent) — and the
Monte-Carlo ATE. Frozen data/ite-observational/{obs.csv,truth.json} + CLI,
mirroring the other portfolio DGPs.

The demo — notebooks/ite_observational.py

An all-CI S-learner (every child node a joint complex intercept over its
parents) fitted once on 20k observational rows, then queried per individual:

  1. the naive observational contrast is biased (−0.671 vs true ATE −0.576) —
    confounding by X1,X2;
  2. L1 sanity: fitted marginals match the data;
  3. L3 ITEs via abduction + do(Tr=0/1) (recomputing the mediators under each
    counterfactual treatment) — recovers the true ITEs at r ≈ 0.99, MAE 0.045,
    ATE −0.575 vs true −0.573;
  4. the learned heterogeneity (effect modified by X2, X3) matches the DGP.

Tests — tests/test_ite_observational.py

Generator pinning (reproducibility, frozen-CSV contract, the TRAM
latent identities as KS-logistic, h_y round-trip, treatment rate, ITE_median
definition, null-scenario sanity, ATE pin) — 10 fast tests — plus a slow
all-CI fit/ITE-recovery test (ATE within 0.05, ITE corr > 0.8).

uv run pytest tests/test_ite_observational.py tests/test_paper_dgps.py -q -m "not slow"
→ 35 passed. Notebook verified headless end-to-end (figure attached in the
description above is reproduced by the .py).

CHANGELOG 0.3.1 + README/notebooks docs updated.

🤖 Generated with Claude Code

oduerr and others added 3 commits June 26, 2026 11:00
Port the 7-variable mediation SCM from mikekr97/MA_Mike
(code/ITE_observational_simulation.R) into the simulations portfolio as
`ITEObservational` (registry key `ite-observational`): a binary treatment
confounded by X1,X2, two Colr mediators X5->X6, and a continuous outcome Y with
a nonlinear baseline h_y(y)=tan(y/2)/0.2 and (X2,X3)*Tr interactions. Four
scenarios toggle main/interaction effects; the generator exposes per-individual
ground truth (ITE_true at the observed latent, ITE_median at the median latent)
and the MC ATE. Frozen data/ite-observational/ CSV (scenario 1) + CLI, mirroring
vaca/carefl.

notebooks/ite_observational.py: an all-CI S-learner TRAM-DAG fitted once on
observational data recovers per-individual ITEs via abduction + do (including
the mediated X5->X6->Y path and the treatment heterogeneity), validated against
the known truth (r ~ 0.99, ATE within ~0.01) and contrasted with the biased
naive contrast.

tests/test_ite_observational.py: reproducibility, frozen-CSV contract, the TRAM
latent identities (KS-logistic), h_y round-trip, treatment rate, ITE_median
definition, null-scenario sanity, ATE pin (fast); plus a slow all-CI fit/ITE
recovery test. CHANGELOG 0.3.1 + README/notebooks docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The DGP was attributed to "Krause", a surname I inferred from the GitHub handle
mikekr97 rather than any source — it is not verified. Attribute instead to the
verifiable master's-thesis repo mikekr97/MA_Mike.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…l-dgp

# Conflicts:
#	CHANGELOG.md
#	notebooks/README.md
@oduerr oduerr merged commit d058199 into main Jun 26, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant