Causal Representation Learning: Why Observational Data Alone Cannot Identify Latent Causes

Most machine learning models learn representations of what is observed. Causal representation learning asks for something harder: representations of the underlying causes that generated the observations, recovered up to a transformation we can actually interpret and intervene on. The distinction matters because a representation that predicts well can still be useless, or actively misleading, when the world changes.

Consider images of a scene. The pixels are observed. The latent causes might be the position of an object, its colour, the lighting, and the camera angle. A standard autoencoder can compress the pixels into a compact code that reconstructs them well. That code, however, need not line up with the true causes at all. It might mix position and lighting into a single coordinate, split one cause across many coordinates, or encode a spurious shortcut. Reconstruction is not identification.

From representation learning to causal representation

In ordinary representation learning, we learn a map from observations X to a representation Z, and often back again. Success is usually judged by downstream prediction, reconstruction error, or clustering quality. None of these criteria pins down Z uniquely. Many different representations can achieve the same predictive performance while disagreeing completely about what causes what.

Causal representation learning adds structure. It assumes the observations are generated by latent variables with causal relationships among them, plus a mixing function that turns those latents into observations. Schematically:

Here Z collects the latent causal variables, g is the (usually nonlinear) mixing function, and X is what we see. The goal is to recover Z, or at least a version of it that preserves its causal structure, from X alone or from X together with extra information such as interventions, multiple environments, or time structure.

The phrase "up to" is doing most of the work. We rarely hope to recover Z exactly, with the same scale, sign, and coordinate labels. Instead we aim for identifiability up to a class of transformations that do not change the causal meaning: permuting the coordinates, rescaling them, or applying a monotone transformation to each coordinate separately. If two representations differ only in those ways, they carry the same causal information. If they differ by a rotation that mixes coordinates, they generally do not.

Why observational data alone is not enough

The core obstacle is a classical non-identifiability result for nonlinear independent component analysis. Suppose the latent variables are independent and the mixing function g is nonlinear and invertible. Then, from the distribution of X alone, we cannot uniquely recover Z. For any candidate solution, we can construct another one that produces exactly the same distribution over X but mixes the latents in a way that destroys their individual meaning.

Intuition helps here. Imagine two independent latents, the size of an object and its brightness, passed through a flexible nonlinear warp into pixels. A learner could recover a rotated, entangled version of size and brightness that still explains every pixel distribution perfectly. Observational data cannot adjudicate between the true latents and the entangled impostor, because both generate identical observations. This is not a failure of optimisation or of having too little data. With infinite observational data, the ambiguity remains.

Formally, the observational distribution p(X) constrains g and the distribution of Z only jointly. Many pairs (g, p(Z)) induce the same p(X). Without an extra assumption or an extra source of variation, the decomposition is underdetermined. This is the same reason that correlation alone cannot identify a causal graph: different causal structures can be observationally equivalent.

A concrete failure mode follows. Suppose a representation entangles a robust cause, such as object shape, with a spurious correlate, such as background colour. A classifier built on that representation may perform well in the training distribution and fail badly when the background changes. The model did not learn the wrong predictor by accident; from observational data alone, it had no way to tell which part of the entangled code was the cause and which part was the correlate.

What breaks the symmetry

Identifiability returns when the data contain variation that acts differently on different latents. Three kinds of auxiliary information are especially powerful.

First, interventions. If we can intervene on one latent variable while holding the others fixed, or observe the system after such an intervention, the entangled alternatives start to disagree. An intervention on position should change the representation coordinate for position, not the coordinate for lighting. Interventional data therefore provide a fingerprint that separates causes that observational data conflates.

Second, multiple environments. Suppose the same causal mechanism operates in several environments, but the distribution of one latent shifts across environments while others stay fixed. That selective shift is informative. A representation that entangles the shifting latent with a stable one will show a pattern of change that the true representation does not, and vice versa. Methods built on this idea, including invariant causal prediction and its representation-learning extensions, exploit heterogeneity across environments as a substitute for explicit interventions.

Third, temporal or grouped structure. If latents evolve over time with independent dynamics, or if observations come in pairs that share some latents but not others, that structure constrains the mixing function. For example, in a video, the identity of an object may stay constant across frames while its position changes. A representation that respects that grouping is favoured over one that mixes identity and position arbitrarily.

In each case the logic is the same: extra variation must be rich enough, and structured enough, to rule out the entangled alternatives. Weak or unstructured variation does not suffice. A single intervention on all latents at once, or environments that shift everything together, leaves much of the ambiguity intact.

A worked example: two latents, one entanglement trap

Consider a toy generative process with two independent latents: the angle of a light source, call it A, and the rotation of an object, call it R. Images X are a nonlinear function of both:

A learner trained only to reconstruct X might recover coordinates U and V defined by a nonlinear mixing of A and R, for instance U = A + R and V = A - R in some transformed space. Reconstruction from (U, V) can be perfect, and a downstream task that only needs the pair (A, R) jointly may still work. But an intervention that changes only the light angle A will change both U and V. A practitioner who interprets U as "lighting" and V as "rotation" will draw the wrong conclusion about what changed and why.

Now add interventional data: images taken after moving only the light, and separately after rotating only the object. The true representation predicts a clean pattern, one coordinate moves under each intervention. The entangled representation predicts that both coordinates move under each intervention. The data now distinguish the two, and the learner can be forced toward the representation whose coordinates respond selectively. That selective response is the operational signature of a causal representation.

A minimal code sketch makes the trap concrete. In a simulation, generate latents, mix them nonlinearly, and check whether a learned representation responds selectively to single-latent interventions:

import numpy as np

rng = np.random.default_rng(0)
n = 10000

# True latents: light angle and object rotation
A = rng.uniform(-1, 1, size=n)
R = rng.uniform(-1, 1, size=n)

# A nonlinear mixing into observations (a stand-in for pixels)
X1 = np.sin(A + R) + 0.3 * A ** 2
X2 = np.cos(A - R) + 0.3 * R ** 2
X = np.column_stack([X1, X2])

# An entangled representation: both coordinates mix A and R
U = A + R
V = A - R

# Intervene only on A: both U and V move, so neither is a pure cause
A_intervened = A + 0.5
print("U moves under A-only intervention:",
      np.mean(np.abs((A_intervened + R) - U)) > 0)
print("V moves under A-only intervention:",
      np.mean(np.abs((A_intervened - R) - V)) > 0)

The point is not the particular mixing function. It is that reconstruction and prediction checks pass for the entangled code, while intervention checks fail. Evaluating a causal representation therefore requires evaluating it under change, not just on held-out samples from the same distribution.

What identifiability results actually guarantee

Modern identifiability theory for causal representation learning is careful about its assumptions, and that care is a feature. Typical results assume something like: the mixing function is invertible and smooth, the latents have a known causal graph or independent mechanisms, the auxiliary variable (intervention target, environment label, or time index) influences the latents in a sufficiently varied way, and the learner's model class contains the truth. Under those conditions, the latents can be recovered up to permutation and component-wise transformation.

Each assumption can fail in practice, and the failures are instructive:

• If the mixing function is not invertible, information about some latent may be lost entirely, and no method can recover it from X.
• If the auxiliary variable affects several latents in lockstep, the variation does not separate them.
• If the true causal graph among latents is unknown and dense, recovering individual latents is harder than recovering their joint structure.
• If the model class is too flexible, it can fit the auxiliary structure with an entangled representation anyway, unless regularisation or architectural constraints rule that out.

A common pitfall is to claim a causal representation because a model disentangles on a synthetic benchmark. Disentanglement on data where the latents were generated independently and the mixing is simple does not imply identifiability on real data where latents are causally related, confounded, or observed through a lossy sensor. Benchmark success is evidence about that benchmark, not a general identification guarantee.

Another pitfall is confusing statistical independence with causal separation. Independent latents are easier to identify, but real causes are often dependent through a common cause or a causal chain. Methods that force independence can therefore distort the very structure they aim to recover. The goal is causal identifiability, which is weaker than exact recovery and stronger than mere predictive usefulness, and it must be judged against the right equivalence class.

Limitations and open problems

Causal representation learning is still closer to a research programme than a routine tool. Real interventions are expensive or impossible in many domains, environments are rarely labelled cleanly, and the mixing function from latents to high-dimensional observations such as images or text is far from the invertible smooth maps assumed in theory. Partial identifiability, where only a subset of latents or only their causal graph is recovered, is often the realistic target.

There are also open questions about evaluation. Without ground-truth latents, how do we know a representation is causal? Interventional benchmarks, counterfactual consistency checks, and robustness under distribution shift all help, but none is a complete substitute for knowing the true generative process. Practitioners should treat claims of "causal representation" with the same scepticism they would apply to any causal claim: ask what variation identifies it, what equivalence class is claimed, and what would falsify it.

Takeaway

Observational data tell us what tends to go together. They do not, on their own, tell us which latent factors are causes, how they are mixed into observations, or how the system will respond to an intervention. That gap is not a technical nuisance; it is a fundamental limit of learning from a single distribution. Causal representation learning closes the gap only by bringing in extra structure: interventions, environments, or time that move latents selectively. When that structure is present and its assumptions are honest, we can recover causes up to a transformation that preserves their meaning. When it is absent, the honest conclusion is that the representation is predictive, not causal, and should not be used as if it were.

Connect with me on LinkedIn: linkedin.com/in/umeshtharukaofficial