PCA on independent but non-identically distributed multinormal data

Question

In the empirical phase diagram approach the data are given by a set ${\boldsymbol X}$ of input (controlled) variables (such as pH and temperature) and a set ${\boldsymbol Y}$ of output variables. The empirical phase diagram aims to visualize (at least, roughly) the relation between the input and the output.

The strategy runs as follows in case when there are only two input variables:

• firstly we ignore the input ${\boldsymbol X}$ and we run a PCA on ${\boldsymbol Y}$

• keep the first three principal components

• transform the three coordinates on the principal components into a color using the red-green-blue coding

• plot the color in function of the two input variables, and visualize

I am rather new in PCA analysis (so please do not hesitate to tell me my questions are stupid if they are). I know PCA has a better interpretation when it is applied with a sample of i.i.d. random multivariate normal variables. But I don't know what are the possible pitfalls when this assumption does not hold.

Assume for instance a multivariate regression model for which the distribution of a single multivariate response ${\boldsymbol Y}$ is assumed to be: $${\boldsymbol Y}_i = f({\boldsymbol X}_i) + \epsilon_i \quad \textrm{with } \epsilon_i \sim {\cal N}({\boldsymbol 0}, \Sigma).$$ I think that we ideally should run the PCA on the centered responses ${\boldsymbol Y}_i - \hat{f}({\boldsymbol X}_i)$ in such a situation.

So what are the possible pitfalls if we run the above strategy in such a situation ?

Answer 1

If you run PCA on ${\bf Y}_i - \hat f({\bf X}_i)$, you are visualizing residuals $\epsilon_i$, not the original data ${\bf Y}_i$. In other words, you will analyze the conditional variance of ${\bf Y}_i$, and that may be of interest per se. If it is the variability of ${\bf Y}_i$ that you want to visualize, and you know these outcomes are affected by ${\bf X}_i$'s, then what you are describing appears a moderately sensible visualization, with the caveat that 5D graphs are difficult to read and interpret. You could also look into principal curves if the dependence on ${\bf X}_i$'s is heavily non-linear.

Answer 2

I'm not sure if I agree with your statement about better interpretation for PCA when applied to iid variables. Normality helps because for normal rv's, variance is the natural measure of dispersion, but the iid condition isn't really necessary. When you don't have normal random variables, you can try Hubert et al's Robust PCA for skewed data: Hubert et al.: "Robust PCA for skewed data and its outlier map"