# PCA on independent but non-identically distributed multinormal data

In the empirical phase diagram approach the data are given by a set ${\boldsymbol X}$ of input (controlled) variables (such as pH and temperature) and a set ${\boldsymbol Y}$ of output variables. The empirical phase diagram aims to visualize (at least, roughly) the relation between the input and the output.

The strategy runs as follows in case when there are only two input variables:

• firstly we ignore the input ${\boldsymbol X}$ and we run a PCA on ${\boldsymbol Y}$

• keep the first three principal components

• transform the three coordinates on the principal components into a color using the red-green-blue coding

• plot the color in function of the two input variables, and visualize

I am rather new in PCA analysis (so please do not hesitate to tell me my questions are stupid if they are). I know PCA has a better interpretation when it is applied with a sample of i.i.d. random multivariate normal variables. But I don't know what are the possible pitfalls when this assumption does not hold.

Assume for instance a multivariate regression model for which the distribution of a single multivariate response ${\boldsymbol Y}$ is assumed to be: $${\boldsymbol Y}_i = f({\boldsymbol X}_i) + \epsilon_i \quad \textrm{with } \epsilon_i \sim {\cal N}({\boldsymbol 0}, \Sigma).$$ I think that we ideally should run the PCA on the centered responses ${\boldsymbol Y}_i - \hat{f}({\boldsymbol X}_i)$ in such a situation.

So what are the possible pitfalls if we run the above strategy in such a situation ?

• +1 I think this is a good question not a stupid one. Sep 18, 2012 at 11:01

If you run PCA on ${\bf Y}_i - \hat f({\bf X}_i)$, you are visualizing residuals $\epsilon_i$, not the original data ${\bf Y}_i$. In other words, you will analyze the conditional variance of ${\bf Y}_i$, and that may be of interest per se. If it is the variability of ${\bf Y}_i$ that you want to visualize, and you know these outcomes are affected by ${\bf X}_i$'s, then what you are describing appears a moderately sensible visualization, with the caveat that 5D graphs are difficult to read and interpret. You could also look into principal curves if the dependence on ${\bf X}_i$'s is heavily non-linear.
• IF the data are iid normals, then you get a sphere (the covariance matrix is $\sigma^2I$). With normals, you always get ellipsoids (in a smaller dimension perhaps, because of non-zero correlations.) Sep 20, 2012 at 3:24
• the data are always multivariate. With i.i.d. multinormal ${\cal N}({\boldsymbol \mu}, \Sigma)$ one gets the ellipsoid associated to $\Sigma$, which is a sphere when $\Sigma=\sigma^2 I$. This point is clear. Sep 20, 2012 at 5:11