How can you relate the coefficients of a multivariate regression of ${\bf Y} \sim {\bf X}$ to the coefficients of ${\bf X} \sim {\bf Y}$?

Question

Is anyone aware of a orthogonal multiple regression library that is implemented in say R, Scipy, Matlab, Octave, etc.? (Or even fortran/C...) If I'm not mistaken, it would not be difficult to write but just wanted to check.

Second question: my actual application is for a multivariate regression case where I have two matrices, $\mathbf{Y}$ and $\mathbf{X}$. If I am able to find a total least squares estimator for $\boldsymbol\beta$ and $\boldsymbol\beta'$, would the inverse of its transposed elements equal one another ($\hat{\beta}_{ji}^{-1} = \hat{\beta}'_{ij}$) if the two coefficient matrices are defined by $\mathbf{Y} = \hat{\boldsymbol{\beta}} \mathbf{X}$ and $\mathbf{X} = \hat{\boldsymbol{\beta}}' \mathbf{Y}$? It seems that this equality ($\hat{\beta}_{ji}^{-1} = \hat{\beta}'_{ij}$) does not necessarily hold as it does for the classic case, where the scalar values of $\beta^{-1}=\beta'$ when $\mathbf{Y}$ and $\mathbf{X}$ are simply vectors rather than matrices, but also wanted to check if this was a known fact.

Thanks.

Answer 1

Perhaps I do not understand your question, but the result does not hold even for scalars. Suppose $$ Y=\beta X + \varepsilon, $$ where $Y$, $X$, and $\varepsilon$ are scalar random variables. Suppose $E[X \varepsilon ] = 0$. Then OLS of $Y$ on $X$ converges to $\beta$. But OLS of $X$ on $Y$ does not converge to $1/\beta$. Although it is true that $$ X = (1/\beta) Y + \eta $$ where $\eta = -\varepsilon/\beta$, notice that $E[Y \eta] \neq 0$. That is, $Y$ is correlated with $\eta$ (by definition of the first regression model), and hence OLS of $X$ on $Y$ is inconsistent; that is, it will not converge to anything, let alone $1/\beta$.