Paper: A Unifying Review of Linear Gaussian Models by Roweis & Ghahramani
The generative model is the typical state space model written as \begin{align} \text{state transition equation: }{\bf x}_t &= {\bf A} {\bf x}_{t-1} + {\bf w}_t && {\bf w}_t \sim \mathcal{N} \left( {\bf 0}, {\bf Q} \right) \\ \text{observation equation: }{\bf y}_t &= {\bf C} {\bf x}_t + {\bf v}_t && {\bf v}_t \sim \mathcal{N} \left( {\bf 0}, {\bf R} \right) \end{align} where ${\bf A}$ is the $k \times k$ state transition matrix and ${\bf C}$ is the $p \times k$ observation matrix.
In the paper on. page 2, the authors write
Notice that there is degeneracy in the model: all of the structure in the matrix $\bf Q$ can be moved into the matrices $\bf A$ and $\bf C$. This means we can without loss generality work with models in which $\bf Q$ is the identity matrix.
There is a footnote associated with the passage and it reads
In particular, since is it a covariance matrix, $\bf Q$ is symmetric positive semi-definite and thus can be diagonalized to the form $\bf E \Lambda E^{\top}$ (where $\bf E$ is the rotation matrix of eigenvectors and ${\bf \Lambda}$ is a diagonal matrix of eigenvalues). Thus for any model in which $\bf Q$ is not the identity matrix, we can generate an exactly equivalent model using a new state vector ${\bf x}' = {\bf \Lambda}^{-1/2} {\bf E}^{\top} {\bf x}$ with ${\bf A}' = ( {\bf \Lambda}^{-1/2} {\bf E}^{\top} ) \, {\bf A} \, ( {\bf E} {\bf \Lambda}^{1/2} )$ and ${\bf C}' = {\bf C} ( {\bf E} {\bf \Lambda}^{1/2})$ such that the new covariance of ${\bf x}'$ is the identity matrix: ${\bf Q}' = {\bf I}$.
Question: How is the new covariance of $\bf x'$ equal to the identity matrix?
\begin{align} \text{Var} ({\bf x}') = {\bf \Lambda}^{-1/2} {\bf E}^{\top} \text{Var} ({\bf x}) {\bf E} {\bf \Lambda}^{-1/2} \end{align} equals the identity matrix if $\text{Var} ({\bf x})$ is the identity matrix. What in the model tells use that this is the case?
Update Mar 27: I believe the authors are referring to the conditional covariance in the footnote.
From the generative model we can see that the conditional distribution of ${\bf x}_t \, \vert \, {\bf x}_{t-1}$ is Gaussian with conditional mean ${\bf A} {\bf x}_{t-1}$ and conditional covariance ${\bf Q}$.
Using the transformed state vector, the new generative model is now \begin{align} \text{new state transition equation: }{\bf x}'_t &= {\bf A}' {\bf x}'_{t-1} + {\bf w}'_t && {\bf w}'_t \sim \mathcal{N} \left( {\bf 0}, {\bf I} \right) \\ \text{new observation equation: }{\bf y}_t &= {\bf C}' {\bf x}'_t + {\bf v}_t && {\bf v}_t \sim \mathcal{N} \left( {\bf 0}, {\bf R} \right). \end{align}
Thus, the conditional distribution of ${\bf x}'_t \, \vert \, {\bf x}'_{t-1}$ is Gaussian with conditional mean ${\bf A}' {\bf x}'_{t-1}$ and conditional covariance ${\bf I}$.
