Partial derivative of a linear regression with correlated predictors

Question

Let's set up the situation of having some $Y$ that I think depends on a linear combination of $X_1$ and $X_2$. I could fit a regression model:

$$y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2}$$

We could write this as a function of the predictor variables:

$$y(x_1, x_2) = \beta_0 + \beta_1x_{1} + \beta_2x_{2}$$

Then we would interpret the coefficients as being the partial derivatives.

$$\dfrac{\partial y}{\partial x_1} = \beta_1$$

$$\dfrac{\partial y}{\partial x_2} = \beta_2$$

This is consistent with our usual idea that, as we increase $x_1$ by one unit and leave $x_2$ alone, $y$ changes by $\beta_1$.

However, what if $X_1$ and $X_2$ are correlated? In that case, if we increase $x_1$ by one unit, $x_2$ should change by some amount.

I've gotten as far as thinking that it has something to do with the inner product of $\big(\beta_1, \beta_2\big)$ with itself with respect to the covariance matrix of $X_1$ and $X_2$:

$$\begin{pmatrix} \beta_1 & \beta_2 \end{pmatrix} \begin{pmatrix} \sigma_{1,1} & \sigma_{1,2}\\ \sigma_{2,1} & \sigma_{2,2} \end{pmatrix} \begin{pmatrix} \beta_1 \\ \beta_2 \end{pmatrix} $$

Thoughts? How does the intercept play into this? Certainly the intercept should drop out, but where?

(There should be $\widehat{\text{hats}}$ all over the place, yes.)

Answer 1

The coefficients in a multiple linear regression are by definition conditional coefficients. If you want the marginal relationship, the general answer is to integrate over the distribution of $x_1$ and $x_2$. However, typically, the distribution is unspecified, and people use the empirical distribution instead. In the linear regression case, I think this reduces to simply fitting the model of one variable without the other.