# Does multi-response linear regression introduce artificial correlation among response residuals?

This is a multi-response linear regression experiment. I generate a random matrix of gaussian noise $\mathcal{N}(0,1)$ sized 10x6 called $Y_N$. I then generate an independent response $X$ via either uniform or gaussian noise, 10x1. I then regress $X$ on $Y_N$, generating $\hat{Y}$. I obtain my residuals, $Y_{res}$, by subtracting $\hat{Y}$ from $Y_N$. Standard linear regression.

Since $Y_N$ is 10x6 we have 6 response vectors, therefore $Y_{res}$ also has 6 vectors each containing the residuals corresponding with each response in $Y_N$. I find that the correlation among the columns of $Y_{res}$ is greater than the correlations among the columns of $Y_N$. This is true for >95% of the random experiments performed. Indeed, it looks like the regression is introducing artificial correlation, but why? Is there any way to overcome this limitation? Extending this experiment to multivariate regression, (e.g. X 10x2), further exacerbates the issue.

To be more specific, I compute the correlation among the columns of $Y_N$ ($Y_{res}$) as follows:

Average Square Correlations:

C = covariance(YN)    % This is a square symmetric covariance matrix.

D = C.^2 % Square each cell so that we only have positive numbers

SC(i,j) = D(i,j)/var(i)*var(j)

% i,j refer to respective cells of C, i,j selected so that only the upper triangle of matrix D (omitting diagonal) is included in computation.


(Then take the average of all SC(i,j)'s to obtain average squared correlation.)

-