LiNGAM - Discovery of non-gaussian linear causal models

Brief description

LiNGAM is a new method for estimating structural equation models or linear Bayesian networks. It is based on using the non-Gaussianity of the data.

Several authors (Spirtes et al. 1993; Pearl 2000) have recently formalized concepts related to causality using probability distributions defined on directed acyclic graphs. This line of research emphasizes the importance of understanding the process which generated the data, rather than only characterizing the joint distribution of the observed variables. The reasoning is that a causal understanding of the data is essential to be able to predict the consequences of interventions, such as setting a given variable to some specified value.

One of the main questions one can answer using this kind of theoretical framework is: 'Under what circumstances and in what way can one determine causal structure based on data which is not obtained by controlled experiments but by passive observation only?'. In many cases it is impossible or too expensive to perform controlled experiments, and hence methods for discovering likely causal relations from uncontrolled ('observational') data would be very valuable.

We have developed a method, abbreviated LiNGAM, for identifying Linear, Non-Gaussian, Acyclic causal Models based on purely observational, continuous-valued data. This method can be seen as an extension of the standard SEM (Structural Equation Model; see, for instance, Bollen 1989) framework. The key aspect of our method is that when the data is non-gaussian it is possible to identify more of the generating structure than is possible in the traditional Gaussian setting. The basic method assumes that (a) there are no hidden confounders, and (b) all (or all but one) of the error terms are non-gaussian. Under these conditions it can be shown that the full generating model can be identified in the limit of an infinite sample.

Original article introducing LiNGAM

S. Shimizu, P. O. Hoyer, A. Hyvärinen, and A. J. Kerminen.
A linear non-gaussian acyclic model for causal discovery.
Journal of Machine Learning Research, 7:2003-2030, 2006.

Estimation methods and code

The model introduced in the paper above can be estimated in different ways:

ICA-based LiNGAM

The estimation method in the original 2006 JMLR paper above is distributed as a complete Matlab/Octave code package. (We also distribute R code as part of the bayeslingam package and as part of the VAR-LiNGAM package.) Also implemented in TETRADR package: pcalg by Kalisch et al. Python code by Taku Yoshioka

Pairwise LiNGAM

A method which considers two variables at a time instead of using conventional ICA algorithms, leading to simple and intuitive estimators, was proposed in:

A. Hyvärinen and S. M. Smith.
Pairwise likelihood ratios for estimation of non-Gaussian structural eauation models.
Journal of Machine Learning Research 14:111-152, 2013.
Matlab code
(See also DirectLiNGAM framework to use these estimators for more than two variable cases.)

Various extensions

The basic model in the 2006 JMLR paper has been later extended in different ways. For an extensive and updated list, see this page.

Here are some of them:

Hidden common causes

S. Shimizu and K. Bollen. 
Journal of Machine Learning Research, 15: 2629-2652, 2014. 
Python code

P. O. Hoyer, S. Shimizu, A. J. Kerminen, and M. Palviainen.
Estimation of causal effects using linear non-gaussian causal models with hidden variables.
International Journal of Approximate Reasoning, 49: 362-378, 2008.
Small Matlab code package.

Time series analysis

A. Hyvärinen, K. Zhang, S. Shimizu, and P. O. Hoyer.
Estimation of a structural vector autoregression model using non-Gaussianity.
Journal of Machine Learning Research, 11:1709-1731, 2010.
R code

Mixtures of Gaussian and non-Gaussian variables

P. O. Hoyer, A. Hyvärinen, R. Scheines, P. Spirtes, J. Ramsey, G. Lacerda, and S. Shimizu. 
Causal discovery of linear acyclic models with arbitrary distributions
Proc. 24th Conf. on Uncertainty in Artificial Intelligence (UAI2008), pp. 282-289, Helsinki, Finland, 2008.

Cyclic model

G. Lacerda, P. Spirtes, J. Ramsey and P. O. Hoyer. 
Discovering cyclic causal models by independent components analysis.
Proc. 24th Conf. on Uncertainty in Artificial Intelligence (UAI2008), pp. 366-374, Helsinki, Finland, 2008.

Nonlinear models

K. Zhang and A. Hyvärinen. 
Proc. 25th Conf. on Uncertainty in Artificial Intelligence (UAI2009), pp. 647-655, Montreal, Canada, 2009. 

P. O. Hoyer, D. Janzing, J. Mooij, J. Peters, and B. Schölkopf.
Nonlinear causal discovery with additive noise models.
Advances in Neural Information Processing Systems 21 (NIPS2008), pp. 689-696, 2009.

Survey articles

S. Shimizu. 
Behaviormetrika, 41(1): 65--98, 2014 (Special Issue on Causal Discovery).

K. Zhang and A. Hyvärinen. 
Nonlinear functional causal models for distinguishing cause from effect
Statistics and Causality: Methods for Applied Empirical Research, pp. 185-202, 2016. 

S. Shimizu
Non-Gaussian structural equation models for causal discovery
Statistics and Causality: Methods for Applied Empirical Research, pp. 153-184, 2016. 

Updated by Shohei Shimizu in March 2017.
Created at the Neuroinformatics group in University of Helsinki around 2006.