ICRL-SM — Modeling Soft Intervention Effects for Implicit Causal Representation Learning

Official implementation of ICRL-SM (Implicit Causal Representation Learning via Switchable Mechanisms).

Paper: Modeling Soft Intervention Effects for Implicit Causal Representation Learning · Machine Learning 115(9), 199 (2026) · Springer

Shayan Shirahmad Gale Bagi¹, Zahra Gharaee², Oliver Schulte³, Mark Crowley¹ ¹ Electrical and Computer Engineering, University of Waterloo · ² Systems Design Engineering, University of Waterloo · ³ School of Computing Science, Simon Fraser University

This repository is a derivative work. It builds directly on ws_crl by Qualcomm AI Research (BSD-3-Clause-Clear) and CausalTriplet (Apache-2.0), and also reuses code from score-based-crl (Apache-2.0) and CITRIS (BSD-3-Clause-Clear). Most of the Python modules here originate upstream. See NOTICE for a file-by-file record and LICENSE for terms.

---

The problem

Existing implicit causal representation learning methods assume access to hard interventions, which sever a variable from its parents entirely. Real interventions are usually soft: they modify a mechanism while leaving the graph intact.

_{(a) Heart rate rises with exercise under normal conditions. (b) A hard intervention — a beta-blocker under controlled conditions — suppresses heart rate entirely, breaking the causal effect of exercise. (c) A soft intervention — the same beta-blocker in an uncontrolled setting where participants may consume caffeine — partially restores variation while still damping the effect. The edge survives; only the mechanism changes.}

That surviving edge is the difficulty. Under a soft intervention the post-intervention variable still depends on its parents, so a model has no clean signal telling it which part of the observed change came from the intervention and which came from ordinary parental influence. Left unmodelled, the solution function routes parental effects through the exogenous variable and identifiability is lost.

The idea

ICRL-SM introduces a causal mechanism switch variable V that explicitly absorbs intervention-induced changes to the structural mechanism, so the exogenous variables no longer have to. Rather than letting E carry both noise and soft-intervention effects, the mechanism becomes

Z_i = s_i(E_i ; E_/i , V)

We call this class of models Augmented Implicit Causal Models (AICMs), formally A = (S, Z, E, V). Expanding the solution function around the pre-intervention state v₀ separates the two contributions cleanly:

s_i(ẽ_i ; e_/i , v)  =       s_i(ẽ_i ; e_/i , v₀)      +          R_i
                        └── genuine causal content ──┘   └─ intervention-induced ─┘

The first term is the intrinsic behaviour of Z_i isolated from unwanted soft-intervention effects; the second is what V is there to capture.

Contributions

1. An implicit causal representation learning method that works directly with soft interventions.
2. The causal mechanism switch variable, modelling the subtle shifts soft interventions induce.
3. Identifiability up to reparameterization under a stated set of assumptions.
4. Strong empirical results on synthetic and real image data, including where those assumptions are only approximately met.

---

Architecture

The model processes pre-intervention observations X, post-intervention observations X̃, and their difference X̃ − X (the intervention displacement). Each encoder outputs the mean (M) and variance (V) of a Gaussian. Sampling gives the pre-intervention exogenous variables E, post-intervention exogenous variables Ẽ, and the switch variable V — the last obtained from the encoding of the difference. E passes through two fully connected layers producing scale and location parameters, which together with Ẽ and V feed the solution functions to yield the post-intervention causal variables Z̃.

The solution function is a location–scale noise model:

z̃_i  =  s_i(ẽ_i ; e_/i , v)  =  ( ẽ_i − ( loc_i(e_/i) + h_i(v) ) ) / scale_i(e_/i)

where loc_i and scale_i are networks over the other exogenous variables and h_i carries the influence of the switch variable. This keeps the mapping a diffeomorphism while separating parental effects, intervention effects, and mechanism shifts.

Generative model

_{Pre- and post-intervention causal variables generate the observations x and x̃. The causal variables come from the exogenous variables via the solution functions. The switch variable v models the soft-intervention effect, which under a linear decoder is observable from x̃ − x. For nonlinear generative processes v is not directly decodable from pixel differences and is inferred implicitly during training — the linear case provides intuition, not a requirement.}

Training objective

Three latent variables — V, E, Ẽ — trained as a VAE with a mean-field posterior q(v,e,ẽ | x,x̃) = q(v | x̃−x)·q(e | x)·q(ẽ | x̃):

log p(x,x̃) ≥ E[ log p(x|e) + log p(x̃|ẽ) + log p(x̃−x|v) ]
             − KLD( q(v|x̃−x)·q(e|x)·q(ẽ|x̃) ‖ p(ẽ|e,v)·p(v)·p(e) )

Computationally this is a standard VAE. The switch variable is inferred through one extra lightweight encoder, so cost scales linearly with latent dimension and batch size and stays comparable to conventional VAE training.

---

Identifiability

Theorem 1. Let M and M' be two latent causal models over a shared observation space and shared intervention targets. Under the assumptions below, the following are equivalent: (1) M and M' assign the same likelihood to interventional and observational data; (2) M and M' are equal up to component-wise reparameterization.

	Assumption	Meaning
A.1	Atomic interventions	Each sample involves an intervention on a single causal variable
A.2	Complete intervention targets	All causal variables are intervened on somewhere in the dataset
A.3	Known intervention targets	The intervention target is observed for each sample
A.4	Counterfactual exogenous variables	Only the exogenous variable of the target changes
A.5	Sufficient action variability	Interventions induce distinguishable effects in latent space
A.6	Diffeomorphic decoder	Decoder and solution functions are smooth and invertible
A.7	Observability of intervention effect	The mechanism shift term can be observed or approximated
A.8	Gaussian latent variables	Latent causal and exogenous variables are multivariate normal
A.9	Affinity of solution functions	The solution functions satisfy an affine structure

These are sufficient, not necessary. The real-world experiments violate several of them and the method still works: non-Gaussian latents do not significantly degrade recovery, decoders that are not strictly diffeomorphic still preserve enough information for latent alignment, and approximate rather than perfectly disjoint supports suffice in finite samples.

---

How ICRL-SM compares

Method	Causal mechanism	Mixing function	Intervention	Learning	Identifiability
CausalDisc	Nonlinear	Full row rank polynomial	Soft	Explicit	Permutation & affine
CauCA	Nonlinear	Diffeomorphism	Soft	Explicit	Assumption-dependent
Linear-CD	Linear	Linear	Hard	Explicit	Permutation
LScale-I	Linear / nonlinear	Linear	Hard / soft	Explicit	Assumption-dependent
ILCM	Nonlinear	Diffeomorphism	Hard	Implicit	Permutation & reparameterization
dVAE	Nonlinear	Diffeomorphism	Hard	Implicit	Permutation & reparameterization
IRSS	Nonlinear	Neural network	None	Explicit	N/A
ICRL-SM (ours)	Nonlinear	Diffeomorphism	Soft	Implicit	Reparameterization

ICRL-SM is the only entry that is implicit and handles soft interventions.

---

Installation

pip install --upgrade pip
pip install -r requirements.txt

The experiment runner scripts additionally need jq to read the JSON configs. A Singularity definition file is provided in icrl.def for cluster use.

Datasets

_{Pre- and post-intervention pairs from the Causal-Triplet benchmark. Note that in the Epic-Kitchens rows the camera angle shifts along with the action — the distribution of causal variables conditioned on viewpoint has changed as a side effect of the intervention. That nuisance variation is exactly what V is designed to absorb.}

Dataset	Source	Scale
Synthetic	Generated at training time	100k train / 10k val / 10k test per graph
ProcTHOR	zenodo.org/record/7813658	100k image pairs, 7 actions, 24 objects, 10k environments
Epic-Kitchens	epic-kitchens.github.io	2,632 image pairs, 97 actions (20 sampled, 283 objects)

For synthetic data, each edge is sampled in a fixed topological order from a Bernoulli with p = 0.5, causal variables are mapped to observation space by a random SO(n) rotation, and the soft intervention perturbs the loc network — pre-intervention weights drawn from N(0,1), post-intervention from N(3,1).

To extract the Epic-Kitchens subset:

bash scripts/run_extract_images.sh

Usage

Experiments are driven by JSON configs in configs/. Set "mode" to train or test inside the config and pass it to the runner:

# Synthetic
bash scripts/run_synthetic_experiments.sh configs/synthetic/icrl.json

# Causal-Triplet (ProcTHOR / Epic-Kitchens)
bash scripts/run_causal_triplet_experiments.sh configs/procthor/icrl.json
bash scripts/run_causal_triplet_experiments.sh configs/epickitchens/icrl.json

A config specifies the model, data, and training schedule. seed accepts a list to sweep several runs, and MAX_JOBS controls parallelism:

{
  "mode": "train",
  "experiment": { "name": "icrl", "expdir": "experiments/synthetic/icrl" },
  "data":  { "dataset": "synthetic", "intervention_type": "soft", "nature_seed": 1 },
  "model": { "encoder": "mlp", "scm": "mlp", "scm_adjacency_matrix": "none" },
  "training": { "epochs": 90, "pretrain_epochs": 10, "batch_size": 64, "seed": 4901 },
  "dim": 4
}

MAX_JOBS=4 bash scripts/run_synthetic_experiments.sh configs/procthor/icrl.json

Configs are provided for every model in the comparison — icrl, ilcm, dvae, betavae, lscale (implicit) and enco, dds, fixed_order (explicit). To average metrics across seeds after a sweep, point the script at the experiment root:

bash scripts/average_tests.sh experiments/causaltriplet-thor/icrl

Test results are written to a test/ folder alongside the checkpoint. You can also invoke the entry points directly — python flow/main.py --model icrl --dataset synthetic ... and python flow/test.py ...; see parse_args.py for the full flag list.

Note: pretrained checkpoints are not distributed with this repository.

---

Results

Causal disentanglement on synthetic data

Ten random 4-variable graphs covering forks, chains and colliders; ten seeds each. DCI disentanglement (D) and completeness (C) on held-out test data.

Graph	β-VAE	dVAE	ILCM	LScale-I	ICRL-SM		β-VAE	dVAE	ILCM	LScale-I	ICRL-SM
	D						C
G1	0.38	0.54	0.71	0.31	0.82		0.51	0.69	0.78	0.72	0.87
G2	0.30	0.72	0.75	0.49	0.83		0.49	0.77	0.80	0.63	0.87
G3	0.28	0.51	0.68	0.63	0.98		0.49	0.56	0.78	0.65	0.98
G4	0.16	0.50	0.65	0.58	0.68		0.38	0.69	0.77	0.75	0.78
G5	0.27	0.44	0.53	0.48	0.42		0.45	0.54	0.66	0.48	0.50
G6	0.52	0.62	0.71	0.40	0.98		0.66	0.69	0.86	0.58	0.98
G7	0.39	0.49	0.71	0.16	0.75		0.70	0.73	0.89	0.63	0.89
G8	0.47	0.54	0.50	0.40	0.59		0.60	0.63	0.62	0.60	0.68
G9	0.30	0.68	0.83	0.32	0.85		0.40	0.76	0.86	0.36	0.87
G10	0.39	0.39	0.52	0.68	0.32		0.53	0.56	0.82	0.73	0.70

ICRL-SM identifies the true causal graph in most cases. It struggles on G5, which is fully connected — causal graphs are usually assumed sparse — and on G10.

What drives disentanglement? Taking G3, the best-identified graph, and varying its data generation:

Edges	Post-intervention mechanism	D	C
Chain	Default	0.98	0.98
Full	Default	0.89	0.89
Default	Significantly different	0.68	0.73
Default	Almost similar	0.85	0.86

Sparsity helps. When the post-intervention mechanism differs too much, the switch variable cannot approximate the intervention and more supervision on V is needed; when it is too similar, there is not enough variability to disentangle at all.

Action and object inference on real images

ResNet-50 encoder, 64×64 inputs, prediction from the intervened causal variable z_i only. Five independent runs per model per dataset — 70 experiments in total.

	Method	Epic-Kitchens action	Epic-Kitchens object	ProcTHOR action	ProcTHOR object
Explicit	ENCO	0.62 ± 0.24	0.06 ± 0.03	0.67 ± 0.19	0.51 ± 0.15
	DDS	0.57 ± 0.19	0.05 ± 0.01	0.51 ± 0.28	0.52 ± 0.29
	Fixed-order	0.49 ± 0.26	0.03 ± 0.01	0.60 ± 0.26	0.57 ± 0.30
Implicit	β-VAE	0.19 ± 0.01	0.04 ± 0.01	0.19 ± 0.02	0.28 ± 0.14
	dVAE	0.53 ± 0.06	0.07 ± 0.02	0.54 ± 0.21	0.64 ± 0.32
	ILCM	0.31 ± 0.07	0.04 ± 0.02	0.48 ± 0.05	0.70 ± 0.05
	ICRL-SM	0.69 ± 0.04	0.07 ± 0.01	0.86 ± 0.08	0.78 ± 0.05

ICRL-SM achieves both the highest mean and the lowest standard deviation. The reduced variance matters as much as the mean: high variance across seeds is what a lack of identifiability looks like, with models settling on unstable or spurious representations rather than recovering the true causal variables.

These datasets do not satisfy the theory's assumptions — Gaussian latents and diffeomorphic decoders both fail here — which makes the result a robustness check as much as a performance one.

Ablations

Full representation vs. intervened variable. Predicting from all of z rather than the intervened z_i degrades performance sharply, suggesting non-intervened latents introduce spurious correlations:

	EK action	EK object	PT action	PT object
all of z	0.16	0.16	0.28	0.40
only z_i	0.86	0.18	0.93	0.82

Smaller backbone (ResNet-18). ICRL-SM still leads on three of four metrics:

Method	EK action	EK object	PT action	PT object
ENCO	0.14	0.03	0.16	0.28
DDS	0.16	0.05	0.34	0.35
Fixed-order	0.14	0.05	0.34	0.38
β-VAE	0.15	0.04	0.20	0.36
dVAE	0.16	0.02	0.15	0.38
ILCM	0.19	0.04	0.15	0.42
ICRL-SM	0.35	0.04	0.40	0.69

A different action subset. Re-sampling a different 20 of the 97 Epic-Kitchens actions (277 objects) confirms the gains are not an artifact of the original sample:

Method	EK action	EK object	PT action	PT object
ENCO	0.69	0.13	0.45	0.53
DDS	0.44	0.09	0.64	0.67
Fixed-order	0.79	0.14	0.65	0.54
β-VAE	0.18	0.06	0.30	0.37
dVAE	0.69	0.17	0.81	0.78
ILCM	0.59	0.14	0.70	0.76
ICRL-SM	0.86	0.18	0.93	0.82

Scalability

_{Causal disentanglement for graphs of 5 to 10 variables, 10 seeds each, ICRL-SM vs ILCM. ICRL-SM leads at every size, but both degrade as variables are added: more variables means more confounding and more ambiguity in the causal relations, and more supervision on V is needed to keep the effects separable.}

---

Repository structure

causal/           SCMs, causal graphs, interventions, ENCO post-hoc graph learning
  scm.py            structural causal models and solution-function parameterisations
  graph.py          learned graph representations (ENCO, DDS, fixed-order)
  enco.py           ENCO graph learning
model/
  lcm/icrl.py       ICRL-SM — the switch variable and its ELBO
  lcm/ilcm.py       ILCM baseline
  lcm/elcm.py       explicit latent causal model baseline
  lcm/l_scale_i.py  LScale-I baseline
  encoder/          MLP, ResNet-pair, slot-attention, image VAE encoders/decoders
  classifier.py     action/object heads for the downstream task
data/             loaders, dataset definitions, preprocessing, synthetic generation
scm/              synthetic SCM generators (box, linear, MLP, quadratic)
flow/
  main.py           training entry point
  test.py           evaluation entry point
  visualize.py      result plotting
register/         model and component registry
utils/            metrics, DCI scores, training utilities
configs/          per-dataset, per-model JSON configs
scripts/          experiment runners

---

Citation

@article{shirahmad2026icrlsm,
  title     = {Modeling Soft Intervention Effects for Implicit Causal Representation Learning},
  author    = {Shirahmad Gale Bagi, Shayan and Gharaee, Zahra and Schulte, Oliver
               and Crowley, Mark},
  journal   = {Machine Learning},
  volume    = {115},
  number    = {9},
  pages     = {199},
  year      = {2026},
  publisher = {Springer},
  doi       = {10.1007/s10994-026-07144-5},
  url       = {https://doi.org/10.1007/s10994-026-07144-5}
}

---

Attribution

This work would not exist without the following open-source releases. If you use this repository, please cite the upstream work as well as ours.

Weakly supervised causal representation learning — the codebase this repository is built on. causal/, model/encoder/, model/lcm/{base,elcm,ilcm,flow_lcm}.py, transforms.py, nets.py, utils/{util,metrics,experiment_utils,dci_metrics}.py are derived from it, and model/lcm/icrl.py extends its ILCM implementation.

@inproceedings{brehmer2022weakly,
  title     = {Weakly supervised causal representation learning},
  author    = {Brehmer, Johann and De Haan, Pim and Lippe, Phillip and Cohen, Taco},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2022}
}

Causal Triplet — the dataset and object-centric encoder side of this work. model/encoder/slotpair.py, procthor/action.py, data/dataset.py, data/utils.py and model/classifier.py are derived from it.

@inproceedings{Liu2023CausalTriplet,
  title     = {Causal Triplet: An Open Challenge for Intervention-centric Causal
               Representation Learning},
  author    = {Liu, Yuejiang and Alahi, Alexandre and Russell, Chris and Horn, Max
               and Zietlow, Dominik and Sch{\"o}lkopf, Bernhard and Locatello, Francesco},
  booktitle = {2nd Conference on Causal Learning and Reasoning (CLeaR)},
  year      = {2023}
}

Score-based causal representation learning — scm/, l_scale_i_utils/ and model/lcm/l_scale_i.py come from its reference implementation.

@article{varici2025score,
  title   = {Score-based Causal Representation Learning: Linear and General Transformations},
  author  = {Var{\i}c{\i}, Burak and Acart{\"u}rk, Emre and Shanmugam, Karthikeyan
             and Kocaoglu, Murat and Tajer, Ali},
  journal = {Journal of Machine Learning Research},
  year    = {2025}
}

CITRIS — causal/enco.py implements ENCO and originates here.

@inproceedings{lippe2022citris,
  title     = {{CITRIS}: Causal Identifiability from Temporal Intervened Sequences},
  author    = {Lippe, Phillip and Magliacane, Sara and L{\"o}we, Sindy and Asano,
               Yuki M and Cohen, Taco and Gavves, Efstratios},
  booktitle = {International Conference on Machine Learning},
  year      = {2022}
}

Slot Attention — implemented in model/encoder/slotpair.py, which reached this repository via Causal Triplet.

@inproceedings{locatello2020object,
  title     = {Object-Centric Learning with Slot Attention},
  author    = {Locatello, Francesco and Weissenborn, Dirk and Unterthiner, Thomas
               and Mahendran, Aravindh and Heigold, Georg and Uszkoreit, Jakob
               and Dosovitskiy, Alexey and Kipf, Thomas},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2020}
}

---

License

Multiple licenses apply — there is no single license covering this repository. See LICENSE for the breakdown and LICENSES/ for verbatim texts.

Component	License
Original ICRL-SM contributions	BSD 3-Clause, © 2024 Shayan Shirahmad Gale Bagi
Code derived from ws_crl	BSD 3-Clause Clear, © 2022 Qualcomm Technologies, Inc.
causal/enco.py (CITRIS)	BSD 3-Clause Clear, © 2022 QUVA-Lab, University of Amsterdam
model/encoder/slotpair.py, procthor/action.py, data/dataset.py, data/utils.py, model/classifier.py	Apache License 2.0 (CausalTriplet)
scm/, l_scale_i_utils/, model/lcm/l_scale_i.py	Apache License 2.0 (score-based-crl)