Generalized Least Squares using Moore Penrose pseudo inverse I'm using GLS to fit a model where some independent variables are strongly correlated. Therefore my covariance matrix is singular. I have found that Moore-Penrose pseudo inverse can be used to find an inverse solution even an matrix is singular. Is it safe to replace the inverse of the covariance matrix with a pseudo inverse of the correlation matrix when using a GLS?
 A: You can compute a solution using the Moore-Penrose inverse in place of the (non-existing) usual inverse. That is known to give a minimum-norm solution. That is, with the linear model in matrix form (assuming iid errors)
$$
   Y = X\beta + \epsilon
$$ (you write generalized least square so presumably the covariance matrix of the error term $\epsilon$ is not of the form $\sigma^2 I$, but everything can be generalized to that case.) Then the least squares solution is $\hat{\beta}= (X^TX)^{-1} X^T Y$, but this only exists if $X^T X$ is invertible. Failing that, the equation system 
$$  X^T X \beta = X^T Y
$$ has infinitely many solutions, and we could pick any of those. But it is argued that when the estimated model is used for prediction,  it is advantageous that $\beta$ is "small", since $\beta$ will also amplify errors. So it is natural to choose the solution of minimal norm $ ||\beta||^2 = \sum_j \beta_j^2$. That solution is found by using the Moore-Penrose inverse. Details can be found here Solve $X^TX b = a$ for $b$ using $XX^T$ for a short and wide matrix $X$. 
The you ask if it is safe to replace the inverse of the covariance matrix with a pseudo inverse of the correlation matrix when using a GLS? Well, it is safe. Denote the Moore-Penrose inverse of $X^TX$ with $(X^TX)^+$ and the minimum norm estimator by $\beta^*$. Then 
\begin{align} \DeclareMathOperator{\C}{\mathbb{Cov}}
   \C \beta^* &=& \C\left\{ (X^TX)^+ X^T Y\right\} \\
              &=& (X^TX)^+ X^T \sigma^2 I X (X^T X)^+ \\
              &=& \sigma^2 (X^TX)^+ X^TX  (X^TX)^+ \\
              &=& \sigma^2 (X^TX)^+
\end{align}
where we in the last line used a basic property of the Moore-Penrose inverse, that $A^+ A A^+ = A^+$.
