1
$\begingroup$

Say we take $\pmb X \pmb \beta$ and partition the columns of $\pmb X$ so that we have $\pmb X \pmb \beta_1$ and $\pmb X_2 \pmb \beta_2$. Why does:

$\pmb y = \pmb X \pmb \beta + \pmb e = \pmb X_1 \pmb \beta_1+ \pmb X_2 \pmb \beta_2 + \pmb e$

I can reason through it geometrically. We're finding the the vector that gets as close to $\pmb y$ as possible in the column space of $\pmb X$. Therefore splitting up the matrix isn't going to change $\hat{ \pmb y}$. Furthermore, the coeficients in $\pmb \beta$ will correspond to the the same columns in the split regression because the solution to the original problem is unique.

So I feel comfortable with why the result is true but I don't know how to prove it beyond that... Is it possible to do this reasonably?

$\endgroup$
1
$\begingroup$

By the definition of matrix multiplication,

$$ X\beta = X_{1,1}\beta_1 + X_{1,2}\beta_2 + \dots + X_{1,p}\beta_{p+q}, $$ which is exactly equal to $$ X_1\beta_1 + X_2\beta_2 = X_{1_{1,1}}\beta_{1_1} + X_{1_{1,2}}\beta_{1_2} + \dots + X_{2_{1,q}}\beta_{2_{q}} $$ where $X_1$ and $X_2$ are submatrices split by column, $\beta_1$ and $\beta_2$ are similar splits of the vector $\beta$, $p$ is the number of columns of $X_1$, and $q$ is the number of columns of $X_2$.

$\endgroup$
1
$\begingroup$

You may write

$ \mathbf{y} = \mathbf{X} \mathbf{\beta} + \epsilon= \begin{bmatrix} \mathbf{X}_1, & \mathbf{X}_2 \end{bmatrix} \begin{bmatrix} \mathbf{\beta}_1 \\ \mathbf{\beta}_2 \end{bmatrix} +\epsilon = \mathbf{X}_1 \mathbf{\beta}_1 + \mathbf{X}_2 \mathbf{\beta}_2 +\epsilon $

where $\mathbf{X}$ is decomposed in its columns, and $\beta$ in its rows (with appropriate dimensions of course, i.e. if $y \in \mathbb{R}^N$, and $\mathbf{X} \in \mathbb{R}^{N \times M}$, then $\mathbf{X}_1 \in \mathbb{R}^{N\times M_1}$ and $\mathbf{X}_2 \in \mathbb{R}^{N\times M_2}$, where $M_1+M_2=M$, and for $\beta \in \mathbb{R}^{M}, \beta_1 \in \mathbb{R}^{M_1}, \beta_2 \in \mathbb{R}^{M_2}$).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.