# How does using PCA speed up supervised learning?

In his popular course, Andrew Ng mentions using PCA to speed up supervised learning (Lecture 14.7). The basic idea is dimensionality reduction, wherein the extremely high-dimensional input features $$\{x^{(1)},\ldots, x^{(k)}\} \in R^{N}$$ are mapped to $$\{z^{(1)},\ldots, z^{(k)}\} \in R^{M}$$, where $$M \ll N$$.

Intuitively, the argument that training for lower-dimensional data should be quicker feels correct, but I could not find a good explanation or proof for this intuition.

My specific question is about the influence of de-correlation of input features that PCA performs towards speeding up learning. My intuition is that decorrelated features should be quicker to learn since the covariance matrix of the transformed $$z^{i}$$ features is a diagonal matrix.

For example, in a neural network, the weights for learning correlation between transformed features $$z^{i}$$ do not require much training since they are already de-correlated.

In other words, suppose I choose not to reduce dimensions after PCA (let's just say the eigenvalues are comparable), my supervised training would still be faster due to de-correlated transformed features. Is this intuition correct?