We usually use PCA as a dimensionality reduction technique for data where cases are assumed to be i.i.d.
Question: What are the typical nuances in applying PCA for dependent, non-i.i.d. data? What nice/useful properties of PCA that hold for i.i.d. data are compromised (or lost entirely)?
For example, the data could be a multivariate time series in which case autocorrelation or autoregressive conditional heteroskedasticity (ARCH) could be expected.
Several related question on applying PCA to time series data have been asked before, e.g. 1, 2, 3, 4, but I am looking for a more general and comprehensive answer (without a need to expand much on each individual point).
Edit: As noted by @ttnphns, PCA itself is not an inferential analysis. However, one could be interested in generalization performance of PCA, i.e. focusing on the population counterpart of the sample PCA. E.g. as written in Nadler (2008):
Assuming the given data is a finite and random sample from a (generally unknown) distribution, an interesting theoretical and practical question is the relation between the sample PCA results computed from finite data and those of the underlying population model.
- Nadler, Boaz. "Finite sample approximation results for principal component analysis: A matrix perturbation approach." The Annals of Statistics (2008): 2791-2817.