Given a sample taken from a distribution concentrated near a $d$-dimensional subspace of a Euclidean space, are there published results on the probability that a PCA (principal component analysis) based dimension estimation will correctly return the true dimension $d$?
The precise answer will depend on the setup, but I'm mildly confident that there are results that attempt to answer some formulations of "How well does a PCA-based dimension estimation work?"
Let me offer one precise formulation of this question.
Let $\mu$ be a probability distribution on $\mathbb R^k$ with a probability density function $f_\mu$ that satisfies:
- Confiement near a subspace. There is a $d$-dimensional vector subspace $\pi \subsetneq \mathbb R^k$ and $\epsilon_\pi>0$ such that $f_\mu(x) = 0$ whenever the distance of $x$ from $\pi$ is greater than $\epsilon_\pi$.
- Embodies the subspace. For some radius $R>0$, there is a constant $C_R$ such that we have $f_\mu(x) > C_R$ whenever $\|x\|<R$ and the distance of $x$ from $\pi$ is less than $\epsilon_\pi$.
Let $X$ be a size $m$ i.i.d. sample drawn from $\mu$. Define the following PCA based dimension estimation: given a tolerance level $\epsilon_{PCA}>0$, whenever the sample $X$ has singular values $\lambda_1 \geq \cdots \geq \lambda_m$ obtained from PCA, we take the estimated dimension to be $d_{PCA} := m - \min \{ j : \lambda_j^2 + \cdots + \lambda_m^2 < \epsilon_{PCA} \}$.
Now, given any (small) $1>\delta>0$, can we compute a number $N$ that depends on the above tolerance levels $\epsilon_\pi, R, \epsilon_{PCA}, \delta$ such that whenever the sample size $m$ exceeds $N$, there is a $1-\delta$ chance that the dimension estimation returns the true dimension $d_{PCA} = d$?
Note that the formulation I proposed can be tweaked within the qualitative meaning of the original question to become another sensible question. For example, the residual $L^2$ error for dimension estimation can be replaced by $\lambda_j < \epsilon_{PCA}$.
