# Partial derivative of a linear regression with correlated predictors

Let's set up the situation of having some $$Y$$ that I think depends on a linear combination of $$X_1$$ and $$X_2$$. I could fit a regression model:

$$y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2}$$

We could write this as a function of the predictor variables:

$$y(x_1, x_2) = \beta_0 + \beta_1x_{1} + \beta_2x_{2}$$

Then we would interpret the coefficients as being the partial derivatives.

$$\dfrac{\partial y}{\partial x_1} = \beta_1$$

$$\dfrac{\partial y}{\partial x_2} = \beta_2$$

This is consistent with our usual idea that, as we increase $$x_1$$ by one unit and leave $$x_2$$ alone, $$y$$ changes by $$\beta_1$$.

However, what if $$X_1$$ and $$X_2$$ are correlated? In that case, if we increase $$x_1$$ by one unit, $$x_2$$ should change by some amount.

I've gotten as far as thinking that it has something to do with the inner product of $$\big(\beta_1, \beta_2\big)$$ with itself with respect to the covariance matrix of $$X_1$$ and $$X_2$$:

$$\begin{pmatrix} \beta_1 & \beta_2 \end{pmatrix} \begin{pmatrix} \sigma_{1,1} & \sigma_{1,2}\\ \sigma_{2,1} & \sigma_{2,2} \end{pmatrix} \begin{pmatrix} \beta_1 \\ \beta_2 \end{pmatrix}$$

Thoughts? How does the intercept play into this? Certainly the intercept should drop out, but where?

(There should be $$\widehat{\text{hats}}$$ all over the place, yes.)

• Yes, please do add the hats in the $y$ and $\beta$s – Tim Mak Apr 8 '20 at 2:53

The coefficients in a multiple linear regression are by definition conditional coefficients. If you want the marginal relationship, the general answer is to integrate over the distribution of $$x_1$$ and $$x_2$$. However, typically, the distribution is unspecified, and people use the empirical distribution instead. In the linear regression case, I think this reduces to simply fitting the model of one variable without the other.