8
$\begingroup$

Suppose we take the classical linear regression model:

$$y_i = \beta_0 + \beta_1 x_i + \epsilon_i$$

Over the years, I have heard so many people say that such an interpretation can be drawn from this model:

  • On average, a one unit increase in $x_i$ "causes" a $\beta_1$ unit increase in $y_i$

However, we are also told that this model can not directly imply this type of "causation". As a matter of fact, there is a whole field of Statistics that studies causality called "Causal Inference" (https://en.wikipedia.org/wiki/Causal_inference).

In general, is there any language that can be used to interpret this kind of model without suggesting any misleading claims about causality?

Improve this question
$\endgroup$
5
  • 14
    $\begingroup$ I have rarely heard anyone use the interpretation you gave. In my experience, people are extremely hesitant to use causal language directly, though their words may imply it. $\endgroup$
    – Noah
    May 9 at 5:05
  • 1
    $\begingroup$ @Stats_noob See here: stats.stackexchange.com/questions/493211/… $\endgroup$
    – markowitz
    May 9 at 6:47
  • 1
    $\begingroup$ A naive reding of the algebra would say that if xi increases by one unit, yi increases by β1 except for some term such as delta(ϵi) .... so the increase would be real. All the problem is in the terminology, the word "causes" .... if xi is a voltge controlled by a knob I can turn and yi a voltage output, "causes" would seem to be fair and appropriate, but we would have to know the context of the equation and the relationship between yi and xi, and wether epsilon is negligible (I assume epsilon is small, such as in calculus class)... $\endgroup$
    – JosephDoggie
    May 10 at 15:40
  • $\begingroup$ wait, what? In what context can the result from regression not be interpreted causally? The RHS is non-stochastic and the LHS is stochastic. $\endgroup$
    – user603
    May 11 at 10:21
  • 1
    $\begingroup$ @user603 You can regress anything against anything, it doesn't mean that X causes Y. The classic example is ice cream consumption being highly correlated with thefts, but you find spurious correlations everywhere. My favourite is this cabbage consumption vs. covid mortality plot: medrxiv.org/content/medrxiv/early/2020/07/17/… $\endgroup$
    – mkt
    May 12 at 10:29

2 Answers 2

Reset to default
20
$\begingroup$

On average, a one unit increase in $x_i$ causes is associated with an increase in $y_i$ of $\beta_1$ units.

Improve this answer
$\endgroup$
6
  • 3
    $\begingroup$ "Association" implies uncertainty, at least to my ear. But you already use "on average" to imply it, so I feel one of these terms is redundant. I'd either simply say "a one unit increase in $x_i$ is associated with an increase of $\beta_1$ units in $y_i$" or, to make the uncertainty more explicit, "on average, a one unit increase in $x_i$ corresponds to an increase of $\beta_1$ units in $y_i$." $\endgroup$
    – Igor F.
    May 10 at 7:13
  • 3
    $\begingroup$ Since regressing $x_i$ on $y_i$ would usually give a coefficient smaller in magnitude than $1/\beta_1$, you might move the average so "a one unit increase in $x_i$ corresponds to an average increase of $β_1$ units in $y_i$." $\endgroup$
    – Henry
    May 10 at 9:24
  • 2
    $\begingroup$ I like both answers to this question, but this one is better as a heuristic considering it's parsimonious wording. I do however agree that Igor's wording is perhaps better to avoid assumptions about this answer. $\endgroup$
    – Shawn Hemelstrand
    May 10 at 16:47
  • 2
    $\begingroup$ A unit increase already implies an increase of $1$. Also, I agree with IgorF and Henry that corresponds is a better fit. $\endgroup$
    – Frans Rodenburg
    May 11 at 9:44
  • 1
    $\begingroup$ Regarding the comment by @IgorF.: The "on average" signifies statistical uncertainty, while "associated with" (or "correlated with") signifies causal uncertainty. They are very different and not at all redundant! $\endgroup$
    – Scriddie
    May 22 at 13:26
20
$\begingroup$

There is a very careful formulation in Gelman, Hill, and Vehtari Regression and Other Stories:

From the data alone, a regression only tells us about comparisons between units, not about changes within units. Thus the most careful interpretation of regression coefficients is in terms of comparisons, for example [...] "Comparing two items $i$ and $j$ that differ by an amount $x$ on predictor $k$ but are identical on all other predictors, the predicted difference $y_i - y_j$ is $\beta_k x$, on average.

This is of course a bit of a mouthful.

Improve this answer
$\endgroup$
1
  • 6
    $\begingroup$ I like this formulation better than the one in mkt's answer. This one steers away from the notion of changes in $x_i$, the use of which tempts the reader to think about it causally (as in $P(\cdot|\text{do}(x))$ rather than $P(\cdot|x)$). $\endgroup$
    – Richard Hardy
    May 9 at 9:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.