# Neural Network Matrix Properties

Consider an input layer $$x$$ ($$D$$ dimensional) , a hidden layer $$y$$ $$K dimensional) and an output layer $$z$$ ($$D$$ dimensional) . Let the input to hidden matrix be $$P$$ and the hidden to output matrix be $$Q$$.

If we are using the neural network as an auto encoder then the error is the squared difference between $$z$$ and $$x$$. It can be shown that once the weights have converged , we can express $$P$$ and $$Q$$ as

$$P=(Q^TQ)^{-1}Q^T$$ and $$Q=C_xP^T(PC_xP^T)^{-1}$$

where $$C_x=\displaystyle\sum_n x_nx_n^T$$ and $$n$$ indexes the data points.

Question: If $$Q$$ has rank $$K$$, then for any invertible $$K \times K$$ matrix $$A$$, the matrices $$P_*=AP$$ and $$Q_*=QA^{-1}$$ also minimise the error. How do I show that we can always find a matrix $$A$$ such that $$Q_*^TQ_*=\mathbb{I}$$ and $$P_*=Q_*^T$$?