I'm interested in the Linear Autoencoder(LAE), and I knew that, at convergence point, the subspace LAE learns is the same as the subspace PCA learns up to linear transformations. Also, the loss function has saddle points, and its local minima become global minima. Here, the problem setting is as discussed in "Neural Networks and Principal Component Analysis: Learning from Examples Without Local Minima"(1989)(http://www.vision.jhu.edu/teaching/learning/deeplearning19/assets/Baldi_Hornik-89.pdf)
It seems that those theoretical facts were studied and derived in late 1980s and 1990s because of the computational constraints of those times, and I'm grateful I have those results. However, I'm also interested in its practical side. More concretely, I want to know about the convergence rate and the way that the LAE recovers the principal subspace (i.e. which principal direction is tend to be learned faster than the others) when using the usual SGD algorithm.
Do you know if there are any works related to that topic? Although I found several articles related to that, they focus on different neural networks, not on LAE.