How to simultaneously fit multiple correlated linear models I have two data sets labeled $\alpha$ and $\beta$, with corresponding
models that I want to fit to.
$$
\textbf{Y}_{\alpha} = \textbf{X}_{\alpha}\cdot\vec{\theta}_{\alpha} + \epsilon \\
\textbf{Y}_{\beta} = \textbf{X}_{\beta}\cdot\vec{\theta}_{\beta} + \epsilon
$$
Here the form of the design matrix $\textbf{X}$ is the same for both $\beta$ and $\alpha$. I expect $\theta_{\alpha}$ to be different from $\theta_{\beta}$. However, I know that data sets $\alpha$ and $\beta$ are highly correlated. I want to take advantage of this correlation to simultaneously determine $\theta_{\alpha}$ and $\theta_{\beta}$. Is there a way to do this?
 A: I think I have come up with a solution. 
I think I can group the data in the following way
$$
\vec{\textbf{y}}=
\begin{pmatrix}
\textbf{Y}_{\alpha}\\\textbf{Y}_{\beta}
\end{pmatrix}
=
\begin{pmatrix}
\textbf{X}_{\alpha} & 0\\ 0 & \textbf{X}_{\beta}
\end{pmatrix}\cdot
\begin{pmatrix}
\theta_{\alpha} \\ \theta_{\beta}
\end{pmatrix}
$$
where 
$\vec{\textbf{y}}$ = 
$\{y_{\alpha_1}, ...,y_{\alpha_{N}}, y_{\beta_1},...,y_{\beta_M}\}$
Then my covariance matrix $\Sigma$ is an $(N+M)\times (N+M)$ matrix.
So the optimal fit parameters are
$$
\begin{pmatrix}
\theta_{\alpha} \\ \theta_{\beta}
\end{pmatrix}^{\text{best}}
=
\Bigg(
\begin{pmatrix}
\textbf{X}_{\alpha} & 0\\ 0 & \textbf{X}_{\beta}
\end{pmatrix}^{\dagger}
\Sigma^{-1}
\begin{pmatrix}
\textbf{X}_{\alpha} & 0\\ 0 & \textbf{X}_{\beta}
\end{pmatrix}
\Bigg)^{-1}
\begin{pmatrix}
\textbf{X}_{\alpha} & 0\\ 0 & \textbf{X}_{\beta}
\end{pmatrix}^{\dagger}\Sigma^{-1}\vec{\textbf{y}}
$$
Constructing projection matrices to get $\theta_{\alpha}$ or $\theta_{\beta}$ are easy to see. 
