2
$\begingroup$

The additive assumption means the effect of changes in a predictor on a response is independent of the effect(s) of changes in other predictor(s).

However, with regression, say, with one continuous predictor and a continuous response, the addition of an additional predictor variables can impact the relationship between a predictor and the response.

My naive interpretation of additivity would suggest that the addition of additional predictors does not change the relationship between, say, the first predictor and the response.

Why is this not the case? What exactly does the additive assumption mean?

$\endgroup$
4
$\begingroup$

I think you're conflating two different concepts here.

Additive effects in linear models

Linear regression assumes that the impact of different covariates is additive, so a simple linear model (for two predictors and a response) would look like this:

$$ y = \alpha + \beta_1 x_1 + \beta_2 x_2 + \epsilon$$

As an example, let's say you changed the value of $x_2$ by adding 1, such that $\tilde{x}_2 = x_2 + 1$, then you would have:

$$ \begin{aligned} \tilde{y} &= \alpha + \beta_1 x_1 + \beta_2 \tilde{x}_2 + \epsilon \\ &= \alpha + \beta_1x_1 + \beta_2 (x_2 + 1) + \epsilon \\ &= (\alpha + \beta_1 x_1 + \beta_2 x_2 + \epsilon) + \beta_2 \\ &= y + \beta_2 \end{aligned}$$

So you can see pretty clearly, in a linear model, that it doesn't matter what value $x_1$ takes - the effect of an incremental change in $x_2$ is the same

Adding additional predictors

This is very different from a situation where you were to add a new predictor to your model! When you add a new predictor, it is quite possible that the other coefficients will change.

$\endgroup$
2
$\begingroup$

When you add another predictor, you are building another model and this is likely to result in a change in the coefficients of existing predictors.

Additivity instead implies that for an existing model, the effect of a predictor on a response variable (whether it be linear or non-linear) is not affected by changes in other existing predictors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.