The heterogeneous graphical Granger model (HGGM) for causal inference among processes with distributions from an exponential family is efficient in scenarios when the number of time observations is much greater than the number of time series, normally by several orders of magnitude. However, in the case of "short" time series, the inference in HGGM often suffers from overestimation. To remedy this, we use the minimum message length principle (MML) to determinate the causal connections in the HGGM. The minimum message length as a Bayesian information-theoretic method for statistical model selection applies Occam's razor in the following way: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct. Based on the dispersion coefficient of the target time series and on the initial maximum likelihood estimates of the regression coefficients, we propose a minimum message length criterion to select the subset of causally connected time series with each target time series and derive its form for various exponential distributions. We propose two algorithms-the genetic-type algorithm (HMMLGA) and exHMML to find the subset. We demonstrated the superiority of both algorithms in synthetic experiments with respect to the comparison methods Lingam, HGGM and statistical framework Granger causality (SFGC). In the real data experiments, we used the methods to discriminate between pregnancy and labor phase using electrohysterogram data of Islandic mothers from Physionet databasis. We further analysed the Austrian climatological time measurements and their temporal interactions in rain and sunny days scenarios. In both experiments, the results of HMMLGA had the most realistic interpretation with respect to the comparison methods. We provide our code in Matlab. To our best knowledge, this is the first work using the MML principle for causal inference in HGGM.

Faculty of Computer Science University of Vienna 1090 Wien Austria

Institute of Computer Science of the Czech Academy of Sciences 18207 Prague Czech Republic

s UniVie University of Vienna 1090 Wien Austria

See more in PubMed

Behzadi S., Hlaváčková-Schindler K., Plant C. Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer; Cham, Switzerland: 2019. Granger Causality for Heterogeneous Processes.

Zou H. The adaptive lasso and its oracle property. J. Am. Stat. Assoc. 2006;101:1418–1429. doi: 10.1198/016214506000000735. DOI

Hryniewicz O., Kaczmarek K. Strengthening Links Between Data Analysis and Soft Computing. Springer; Cham, Switzerland: 2015. Forecasting short time series with the bayesian autoregression and the soft computing prior information; pp. 79–86.

Bréhélin L. A Bayesian approach for the clustering of short time series. Rev. D’Intell. Artif. 2006;20:697–716. doi: 10.3166/ria.20.697-716. DOI

Wallace C.S., Boulton D.M. An information measure for classification. Comput. J. 1968;11:185–194. doi: 10.1093/comjnl/11.2.185. DOI

Shimizu S., Inazumi T., Sogawa Y., Hyvärinen A., Kawahara Y., Washio T., Hoyer P.O., Bollen K. DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model. J. Mach. Learn. Res. 2011;12:1225–1248.

Kim S., Putrino D., Ghosh S., Brown E.N. A Granger causality measure for point process models of ensemble neural spiking activity. PLoS Comput. Biol. 2011;7:e1001110. doi: 10.1371/journal.pcbi.1001110. PubMed DOI PMC

Arnold A., Liu Y., Abe N. Temporal causal modeling with graphical Granger methods; Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; San Jose, CA, USA. 12–15 August 2007; pp. 66–75.

Shojaie A., Michailidis G. Discovering graphical Granger causality using the truncating lasso penalty. Bioinformatics. 2010;26:i517–i523. doi: 10.1093/bioinformatics/btq377. PubMed DOI PMC

Lozano A.C., Abe N., Liu Y., Rosset S. Grouped graphical Granger modeling methods for temporal causal modeling; Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Paris, France. 28 June–1 July 2009; pp. 577–586.

Nelder J., Wedderburn R. Generalized Linear Models. J. R. Stat. Soc. Ser. A (General) 1972;135:370–384. doi: 10.2307/2344614. DOI

Hlaváčková-Schindler K., Plant C. Poisson Graphical Granger Causality by Minimum Message Length; Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases 2020 (ECML/PKDD); Ghent, Belgium. 14–18 September 2020.

Granger C.W. Investigating causal relations by econometric models and cross-spectral methods. Econometrica. 1969;37:424–438. doi: 10.2307/1912791. DOI

Mannino M., Bressler S.L. Foundational perspectives on causality in large-scale brain networks. Phys. Life Rev. 2015;15:107–123. doi: 10.1016/j.plrev.2015.09.002. PubMed DOI

Maziarz M. A review of the Granger-causality fallacy. J. Philos. Econ. Reflect. Econ. Soc. Issues. 2015;8:86–105.

Granger C.W. Some recent development in a concept of causality. J. Econom. 1988;39:199–211. doi: 10.1016/0304-4076(88)90045-0. DOI

Lindquist M.A., Sobel M.E. Graphical models, potential outcomes and causal inference: Comment on Ramsey, Spirtes and Glymour. NeuroImage. 2011;57:334–336. doi: 10.1016/j.neuroimage.2010.10.020. PubMed DOI PMC

Spirtes P., Glymour C.N., Scheines R., Heckerman D. Causation, Prediction, and Search. MIT Press; Cambridge, MA, USA: 2000.

Glymour C. Counterfactuals, graphical causal models and potential outcomes: Response to Lindquist and Sobel. NeuroImage. 2013;76:450–451. doi: 10.1016/j.neuroimage.2011.07.071. PubMed DOI

Marinescu I.E., Lawlor P.N., Kording K.P. Quasi-experimental causality in neuroscience and behavioural research. Nat. Hum. Behav. 2018;2:891–898. doi: 10.1038/s41562-018-0466-5. PubMed DOI

Wallace C.S., Freeman P.R. Estimation and inference by compact coding. J. R. Stat. Soc. Ser. B. 1987;49:240–252. doi: 10.1111/j.2517-6161.1987.tb01695.x. DOI

Wallace C.S., Dowe D.L. Minimum message length and Kolmogorov complexity. Comput. J. 1999;42:270–283. doi: 10.1093/comjnl/42.4.270. DOI

Schmidt D.F., Makalic E. Australasian Joint Conference on Artificial Intelligence. Springer; Cham, Switzerland: 2013. Minimum message length ridge regression for generalized linear models; pp. 408–420.

Segerstedt B. On ordinary ridge regression in generalized linear models. Commun. Stat. Theory Methods. 1992;21:2227–2246. doi: 10.1080/03610929208830909. DOI

Computational Complexity of Mathmatical Operations. [(accessed on 2 October 2020)]; Available online: https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations.

Rissanen J. Stochastic Complexity in Statistical Inquiry. Volume 15. World Scientific; Singapore: 1989. p. 188.

Barron A., Rissanen J., Yu B. The minimum description length principle in coding and modeling. IEEE Trans. Inf. Theory. 1998;44:2743–2760. doi: 10.1109/18.720554. DOI

Hansen M., Yu B. Model selection and minimum description length principle. J. Am. Stat. Assoc. 2001;96:746–774. doi: 10.1198/016214501753168398. DOI

Hansen M.H., Yu B. Minimum description length model selection criteria for generalized linear models. Lect. Notes Monogr. Ser. 2003;40:145–163.

Marx A., Vreeken J. Telling cause from effect using MDL-based local and global regression; Proceedings of the 2017 IEEE International Conference on Data Mining; New Orleans, LA, USA. 18–21 November 2017; pp. 307–316.

Marx A., Vreeken J. Causal inference on multivariate and mixed-type data; Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases; Dublin, Ireland. 10–14 September 2018; pp. 655–671.

Budhathoki K., Vreeken J. Origo: Causal inference by compression. Knowl. Inf. Syst. 2018;56:285–307. doi: 10.1007/s10115-017-1130-5. DOI

Hlaváčková-Schindler K., Plant C. Graphical Granger causality by information-theoretic criteria; Proceedings of the European Conference on Artificial Intelligence 2020 (ECAI); Santiago de Compostela, Spain. 29 August–2 September 2020; pp. 1459–1466.

McIlhagga W.H. Penalized: A MATLAB toolbox for fitting generalized linear models with penalties. J. Stat. Softw. 2016;72 doi: 10.18637/jss.v072.i06. DOI

Zou H., Hastie T., Tibshirani R. On the “degrees of freedom” of the lasso. Ann. Stat. 2007;35:2173–2192. doi: 10.1214/009053607000000127. DOI

[(accessed on 5 September 2020)]; Available online: https://meteo.boku.ac.at/wetter/mon-archiv/2020/202009/202009.html.

Zentralanstalt für Meteorologie und Geodynamik 1190 Vienna, Hohe Warte 38. [(accessed on 5 September 2020)]; Available online: https://www.zamg.ac.at/cms/de/aktuell.

Alexandersson A., Steingrimsdottir T., Terrien J., Marque C., Karlsson B. The Icelandic 16-electrode electrohysterogram database. Nat. Sci. Data. 2015;2:1–9. doi: 10.1038/sdata.2015.17. PubMed DOI PMC

[(accessed on 5 September 2020)]; Available online: https://www.physionet.org.

Mikkelsen E., Johansen P., Fuglsang-Frederiksen A., Uldbjerg N. Electrohysterography of labor contractions: Propagation velocity and direction. Acta Obstet. Gynecol. Scand. 2013;92:1070–1078. doi: 10.1111/aogs.12190. PubMed DOI

Agresti A. Categorical Data Analysis. Volume 482 John Wiley and Sons; Hoboken, NJ, USA: 2003. Section 12.3.3.

Huber P.J. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Volume 1. University of California Press; Berkeley, CA, USA: 1967. The behavior of maximum likelihood estimates under nonstandard conditions; pp. 221–233.