2
$\begingroup$

I was having a discussion with someone regarding OR’s estimated trough a logistic regression and then he claims that OR’s for continuous variables can only be estimated trough a logistic regression. AFAIK, OR’s are defined only for categorical variables so I was kind of confused. He then proceeds to claim as well that, in the case of a linear regression, the parameter associated with continuous covariates can also be interpreted as a mean differences.

Let us review the case for a general model with a categorical variable. Let us define $Y$ as our dependent variable and a covariate $W$ which has values $A$ and $B$. If we use a reference codification for W, for example $I_{W=B}$ (i.e: if $W=A$ then $I_{W=B}=0$ and if $W=B$ then $I_{W=B}=1$). Then a general model would be

$$g(E(Y|I_{W=B}=x))=\beta_0+\beta_1x$$

for this model it can be shown that $\beta_0=g(E(Y|I_{W=B}=0))=g(E(Y|W=A))$ and $$\beta_1=g(E(Y|I_{W=B}=1))-g(E(Y|I_{W=B}=0))=$$ $$g(E(Y|W=B))-g(E(Y|W=A))$$

if $g$ is the identity we get a linear (normal) model and $\beta_1$ could be interpreted as a mean difference (MD), if $g=logit$ then $\beta_1$ would be the logarithm of an odds ratio.

In the case the covariate is a continuous variable the model would be

$$g(E(Y|X=x))=\beta_0+\beta_1x$$

let us evaluate the model for a given $x$ and $x+1$ and solve it for $\beta_1$:

$$g(E(Y|X=x+1))=\beta_0+\beta_1(x+1)$$

then $$ g(E(Y|X=x+1)) – g(E(Y|X=x))=\beta_0+\beta_1(x+1) - (\beta_0+\beta_1x)=\beta_1$$ Again for different link functions we would interpret $\beta_1$ as a MD or an OR.

However I have never seen such an interpretation of these parameters in such a way. In most text books (in the common case of identity link) you interpret $\beta_1$ as a rate of change (a derivative), not as a MD. I was arguing that the interpretation of MD or OR for continuous covariates was weird and only valid for increments in one unit in the convariate, breaking the “continuous” nature of the covariate. More over MD and OR make sense and are defined for categorical variables not for continuous ones.

What are your takes on this? Is appropriate to interpret parameters of continuous covariates as if they were effect measures for categorical variables?

$\endgroup$

1 Answer 1

2
+50
$\begingroup$

Let's focus on the linear regression case, as this question is about the interpretation of regression coefficients, not about the role of the link function.

Mathematically, someone is correct. The regression coefficient $\beta_i$ for a continuous predictor $X_i$ is the difference between two (conditional) means, ie. it's a mean difference.

To be precise, let's assume that $X_i$ doesn't appear in any other regression terms: no interactions, polynomials, splines, etc. For clarity I write the rest of the regression equation as $\boldsymbol{\beta}_{-i}\mathbf{X}_{-i}$ to indicate that $\mathbf{X}_{-i}$ doesn't involve $X_i$.

$$ \begin{aligned} & \operatorname{E}\left\{Y | X_i=x_i+1,\mathbf{X}_{-i}=\mathbf{x}_{-i}\right\} -\operatorname{E}\left\{ Y | X_i=x_i,\mathbf{X}_{-i}=\mathbf{x}_{-i}\right\} \\ &= \Big(\beta_i(x+1)-\beta_ix\Big) +\Big(\boldsymbol{\beta}_{-i}\mathbf{x}_{-i}-\boldsymbol{\beta}_{-i}\mathbf{x}_{-i}\Big) \\ &= \beta_i \end{aligned} $$

So we have two statements about $\beta_i$:

  • $\beta_i$ is the change in the expected value of the response $Y$ per unit change in the predictor $X_i$.
  • $\beta_i$ is the difference in the expected $Y$ between units (patients, respondents, items, ...) that differ by one unit of $X_i$ but otherwise are equal in all other predictors.

These statements are equivalent as long as we don't interpret the "unit change in $X_i$" causally.

Here is a fictitious example. Say $X_i$ is number of cups of coffee per day and $Y$ is a (totally objective, universal, continuous measure of) happiness. We have some observational data on happiness and we estimate $\hat{\beta}_i$ = 0.5. This means that, on average, people who drink two cups of coffee per day are 0.5 units happier than people who drink one cup of coffee. It doesn't mean that if a person starts drinking one more cup of coffee a day, they well become 0.5 units happier than before.

I think that the "change in Y per unit change in X" language tends to invite a causal interpretation. That's justified only if we are doing causal analysis.

So even though it's wordier the "difference in means" interpretation might be more helpful.

References

Andrew Gelman and colleagues discuss the two interpretations (causal and predictive) of regression coefficients in their textbooks on regression analysis.

[1] Gelman and J. Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 2007. See Section 3.2.
[2] Gelman, J. Hill, and A. Vehtari. Regression and Other Stories. Cambridge University Press, 2020. See Section 10.2. It's freely available online.

$\endgroup$
5
  • $\begingroup$ so, in the end, both interpretations are valid? the choose between a rate of change or mean difference interpretation is related with the fact whether the study is causal, observational or predictive? would you mind to point out the chapter in which those references discuss the interpretation of parameters? thanks for your feedback $\endgroup$ Sep 11, 2022 at 22:00
  • $\begingroup$ Section 3.2 is in Chapter 3. Section 10.2 in Chapter 10. $\endgroup$
    – dipetkov
    Sep 11, 2022 at 23:23
  • $\begingroup$ I didnt see the info about sections, sorry for that. Although I was reluctant to accept the “mean difference” interpretation for continuous covariates, the references provided are convincing. I would point out that in the case of a logistic regression, it is more easier to interpret that way than as a rate of change, since this interpretation would only be valid in the logit scale. $\endgroup$ Sep 11, 2022 at 23:45
  • $\begingroup$ Also this would suggest to scale a continuous variable to obtain a more meaningful interpretation, for example if I have a time covariate measured in days, for researches would be more meaningful to provide a mean difference not in day units but in months or year units, making interpretation more easy and valuable. $\endgroup$ Sep 11, 2022 at 23:46
  • 1
    $\begingroup$ I agree that scaling predictors is another important aspect of interpretation. I couldn't remember if/where Gelman discusses this in the textbooks, so here is a post on his blog instead. When to standardize regression inputs and when to leave them alone. $\endgroup$
    – dipetkov
    Sep 12, 2022 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.