# What is the intuition behind the regression coefficient matrix in multivariate linear regression model?

Consider the usual multivariate linear regression model where we solve for $\mathbf{\hat{b}^{OLS}}$

We have the equation

$$\mathbf{y}=\mathbf{X}\mathbf{b}+\mathbf{u}$$

where $\mathbf{y}$ is the outcome matrix, $\mathbf{X}$ is the predictor matrix, $\mathbf{b}$ (not sure what to call it) and $\mathbf{u}$ is the error matrix. We assume $\mathbf{X}$ is of dimensions $n \times (k+1)$ where an extra row of $1$s is added to permit a $b_0$ term in each row. Therefore, $\mathbf{b}$ is of dimensions $(k+1) \times 1$

The OLS method solves

$$\underset{\mathbf{b}}\min \mathbf{u}^\text{T}\mathbf{u}$$ $$\text{s.t. }\mathbf{u} = \mathbf{y}-\mathbf{Xb}$$

The solution is

$$[\mathbf{X}^\text{T}\mathbf{X}]^{-1}\mathbf{X}^\text{T}\mathbf{y}=\mathbf{\hat{b}^{OLS}}$$

# My Question

What is the intuition behind this statement of $\mathbf{\hat{b}^{OLS}}$?

Specifically, in the simple linear regression model, $\hat{b}^{OLS}$ was just the slope of the best fit line $\hat{Y} = \hat{b}_0+\hat{b}_1X$ or the intercept. But what about in the multivariate case?

• I would guess this question has been asked before; have you checked? – Richard Hardy May 8 '16 at 6:23

The intuition is very simple. We try to project $Y$ onto the linear subspace spanned by $X_{1}\cdots X_{n}$. This is precisely the best $Y'$ we have such that $|Y-Y'|^2$ has the smallest value possible.