In the empirical phase diagram approach the data are given by a set ${\boldsymbol X}$ of input (controlled) variables (such as pH and temperature) and a set ${\boldsymbol Y}$ of output variables. The empirical phase diagram aims to visualize (at least, roughly) the relation between the input and the output.

The strategy runs as follows in case when there are only two input variables:

  • firstly we ignore the input ${\boldsymbol X}$ and we run a PCA on ${\boldsymbol Y}$

  • keep the first three principal components

  • transform the three coordinates on the principal components into a color using the red-green-blue coding

  • plot the color in function of the two input variables, and visualize

I am rather new in PCA analysis (so please do not hesitate to tell me my questions are stupid if they are). I know PCA has a better interpretation when it is applied with a sample of i.i.d. random multivariate normal variables. But I don't know what are the possible pitfalls when this assumption does not hold.

Assume for instance a multivariate regression model for which the distribution of a single multivariate response ${\boldsymbol Y}$ is assumed to be: $${\boldsymbol Y}_i = f({\boldsymbol X}_i) + \epsilon_i \quad \textrm{with } \epsilon_i \sim {\cal N}({\boldsymbol 0}, \Sigma).$$ I think that we ideally should run the PCA on the centered responses ${\boldsymbol Y}_i - \hat{f}({\boldsymbol X}_i)$ in such a situation.

So what are the possible pitfalls if we run the above strategy in such a situation ?

  • $\begingroup$ +1 I think this is a good question not a stupid one. $\endgroup$ – Michael R. Chernick Sep 18 '12 at 11:01

If you run PCA on ${\bf Y}_i - \hat f({\bf X}_i)$, you are visualizing residuals $\epsilon_i$, not the original data ${\bf Y}_i$. In other words, you will analyze the conditional variance of ${\bf Y}_i$, and that may be of interest per se. If it is the variability of ${\bf Y}_i$ that you want to visualize, and you know these outcomes are affected by ${\bf X}_i$'s, then what you are describing appears a moderately sensible visualization, with the caveat that 5D graphs are difficult to read and interpret. You could also look into principal curves if the dependence on ${\bf X}_i$'s is heavily non-linear.

| cite | improve this answer | |
  • $\begingroup$ There's no conditional distribution in the regression model so I don't see why are you talking about some conditional variance. Moreover I don't want to use any model, my question is about what happens if there is such a model but we don't use it. Thanks for mentioning the principal curves, I didn't know this notion. $\endgroup$ – Stéphane Laurent Sep 19 '12 at 5:02
  • $\begingroup$ Oops sorry I'm still sleeping ;) You mean that the regression model is a modeling of the conditional distribution of Y given the covariates, right. $\endgroup$ – Stéphane Laurent Sep 19 '12 at 5:05

I'm not sure if I agree with your statement about better interpretation for PCA when applied to iid variables. Normality helps because for normal rv's, variance is the natural measure of dispersion, but the iid condition isn't really necessary. When you don't have normal random variables, you can try Hubert et al's Robust PCA for skewed data: Hubert et al.: "Robust PCA for skewed data and its outlier map"

| cite | improve this answer | |
  • $\begingroup$ If the data are i.i.d. multinormal then the principal components define the ellipsoid associated to the covariance matrix; that's why I said the interpretation is better. Thanks for Hubert & al's paper. $\endgroup$ – Stéphane Laurent Sep 19 '12 at 4:57
  • $\begingroup$ IF the data are iid normals, then you get a sphere (the covariance matrix is $\sigma^2I$). With normals, you always get ellipsoids (in a smaller dimension perhaps, because of non-zero correlations.) $\endgroup$ – user765195 Sep 20 '12 at 3:24
  • $\begingroup$ the data are always multivariate. With i.i.d. multinormal ${\cal N}({\boldsymbol \mu}, \Sigma)$ one gets the ellipsoid associated to $\Sigma$, which is a sphere when $\Sigma=\sigma^2 I$. This point is clear. $\endgroup$ – Stéphane Laurent Sep 20 '12 at 5:11
  • $\begingroup$ You still get an ellipsoid with dependent, non-identically distributed normals. The only difference is that the ellipsoid might really live in a smaller dimension. $\endgroup$ – user765195 Sep 21 '12 at 3:08
  • $\begingroup$ I don't see what ellipsoid you are talking about. $\endgroup$ – Stéphane Laurent Sep 21 '12 at 5:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.