I've had no regrets since I left mathematics almost 19 years ago. My last memories of studying it are of sitting in a library at the University of Cambridge, staring at lecture notes full of lifeless equations, struggling and failing to care. But occasionally I read something that reminds me of the beauty that can be found in math, and of the remarkable power of formal analysis. Then I see again what once made her attractive to me, if you'll forgive the metaphor.That happened a few days ago as I read Judea Pearl's Causal inference in statistics: An overview (hat tip to Holden Karnofsky). Everything I know about Pearl I learned from his web site. He works in the computer science department at UCLA and is older than he looks in the picture. He researched advanced electronic devices in the 1960s. He has apparently written the bible on Causality, which is to say, the formal study of what we mean when we talk about variables such as diet and health influencing each other. He just released that overview paper in September, which gives a "gentle introduction" to a subject he appears to have done much to develop.The analysis is beautiful, but not merely that. It is insightful enough about what statisticians in economics, medicine, and other fields do every day that I think the paper should be required reading for graduate students in all fields that use statistics to study causality. Reading the paper may be a particular thrill for me because I entered econometrics untutored, running my first regressions in 2002 for a project with Bill Easterly. I have learned econometrics on the fly, and only gradually grasped what I was really doing. From my point of view, Pearl has systematized a few things I had come to understand while offering a much greater vista.The core idea is this: The bulk of the vast field of statistics is about distributions. You can, for example, graph the probability of rolling a 2 or 3 or 4, etc., in a throw of Monopoly dice; the graph is bell shaped, and is a distribution. Distributions can also be multidimensional, which let us speak precisely about how different variables vary in tandem or independently. From such associations and non-associations, we can attempt to infer causal relationships, but that takes a leap of logic and at least until recently the nature of that leap had not been carefully analyzed. Statisticians have used precise mathematical symbols to speak of distributions, but sloppy words to speak of causality:
The aim of standard statistical analysis, typified by regression, estimation, and hypothesis testing techniques, is to assess parameters of a distribution from samples drawn of that distribution. With the help of such parameters, one can infer associations among variables, estimate beliefs or probabilities of past and future events, as well as update those probabilities in light of new evidence or new measurements. These tasks are managed well by standard statistical analysis so long as experimental conditions remain the same. Causal analysis goes one step further; its aim is to infer not only beliefs or probabilities under static conditions, but also the dynamics of beliefs under changing conditions, for example, changes induced by treatments or external interventions.This distinction implies that causal and associational concepts do not mix. There is nothing in the joint distribution of symptoms and diseases to tell us that curing the former would or would not cure the latter. More generally, there is nothing in a distribution function to tell us how that distribution would differ if external conditions were to change---say from observational to experimental setup---because the laws of probability theory do not dictate how one property of a distribution ought to change when another property is modified. This information must be provided by causal assumptions which identify relationships that remain invariant when external conditions change.These considerations imply that the slogan "correlation does not imply causation" can be translated into a useful principle: one cannot substantiate causal claims from associations alone, even at the population level---behind every causal conclusion there must lie some causal assumption that is not testable in observational studies.As I did in a post on the study of microcredit's impacts, if you have ever tried to talk about how certain variables relate to each other causally, you probably drew a picture with nodes (variables) and arrows to link them, something like this diagram from Pearl's paper:
landholdings => microcredit borrowing => household well-beingand Pearl's:
Associational assumptions, even untested, are testable in principle, given sufficiently large sample and sufficiently fine measurements. Causal assumptions, in contrast, cannot be verified even in principle, unless one resorts to experimental control….This makes it doubly important that the notation we use for expressing causal assumptions be meaningful and unambiguous so that one can clearly judge the plausibility or inevitability of the assumptions articulated. Statisticians can no longer ignore the mental representation in which scientists store experiential knowledge, since it is this representation, and the language used to access it that determine the reliability of the judgments upon which the analysis so crucially depends.This paper should, in both the normative and predictive sense, speed the day when statisticians pay adequate attention to the causal assumptions they make.Postscript: I was stunned to spot this button on Pearl's page:
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.