# Is there a “generalized least norm” equivalent to generalized least squares?

In a standard regression problem

$$\mathbf{y} = \mathbf{X} \beta + \mathbf{e} \ ,$$

the solution to $\beta$ when the system is overdetermined is $\hat{\beta}= \left(\mathbf{X}\Sigma^{-1}\mathbf{X}\right)^{-1}\mathbf{X}^T\Sigma^{-1}\mathbf{y}$, where $\Sigma = \sigma^{-2} \operatorname{Var}(\mathbf{e})$.

The solution to an underdetermined system when $\Sigma\equiv \mathbf{I}$ is $\hat{\beta} = \mathbf{X}^T\left(\mathbf{X}\mathbf{X}^T\right)^{-1}\mathbf{y}$. Can a similar formula to the least squares case be written when $\Sigma \ne \mathbf{I}$ such that $\hat{\beta} = \mathbf{X}^T\Sigma^{-1}\left(\mathbf{X}\mathbf{X}^T\Sigma^{-1}\right)^{-1} \mathbf{y}$?

I have not been able to find any good references on this system and would appreciate if you could suggest any also.

• Could you please clarify what you are asking? I do not see any "formulations" in your question--only formulas--so the meaning of "similar formulation to the least squares case" is not apparent. – whuber Jan 20 '14 at 16:30
• My apologies. I meant formulation in the construction of a formula but did not mean anything more abstract. I have changed the the word to say "formula" rather than "formulation." – hatmatrix Jan 20 '14 at 16:51
• Note that your expression for $\hat{\beta}$ in the line starting with "The solution to..." is incorrect; you need to post-multiply by $y$. – jbowman Jan 20 '14 at 21:47

To customize the result to an underdetermined system, note the derivation of GLS as the best linear unbiased estimator uses the factorization $\Sigma = AA^T$ as follows. Define $Y^* = A^{-1}Y$, $X^*=A^{-1}X$, and $e^*=A^{-1}e$. Then the OLS solution to $Y^* = X^*\beta + e^*$ satisfies the Gauss-Markov requirements, and is consequently the best linear unbiased estimator.
When using the right generalized inverse $C^+ = C^T(CC^T)^{-1}$ to form the estimate $\hat{\beta} = X^+Y$ in the underdetermined case, you can just use $X^*$ and $Y^*$ instead of $X$ and $Y$. Writing it out in terms of $A$ and $X$ gives:
$\hat{\beta} = (A^{-1}X)^T(A^{-1}XX^T(A^{-1})^T)^{-1}A^{-1}Y$