In the empirical phase diagram approach the data are given by a set ${\boldsymbol X}$ of input (controlled) variables (such as pH and temperature) and a set ${\boldsymbol Y}$ of output variables. The empirical phase diagram aims to visualize (at least, roughly) the relation between the input and the output.

The strategy runs as follows in case when there are only two input variables:

  • firstly we ignore the input ${\boldsymbol X}$ and we run a PCA on ${\boldsymbol Y}$

  • keep the first three principal components

  • transform the three coordinates on the principal components into a color using the red-green-blue coding

  • plot the color in function of the two input variables, and visualize

I am rather new in PCA analysis (so please do not hesitate to tell me my questions are stupid if they are). I know PCA has a better interpretation when it is applied with a sample of i.i.d. random multivariate normal variables. But I don't know what are the possible pitfalls when this assumption does not hold.

Assume for instance a multivariate regression model for which the distribution of a single multivariate response ${\boldsymbol Y}$ is assumed to be: $${\boldsymbol Y}_i = f({\boldsymbol X}_i) + \epsilon_i \quad \textrm{with } \epsilon_i \sim {\cal N}({\boldsymbol 0}, \Sigma).$$ I think that we ideally should run the PCA on the centered responses ${\boldsymbol Y}_i - \hat{f}({\boldsymbol X}_i)$ in such a situation.

So what are the possible pitfalls if we run the above strategy in such a situation ?

  • $\begingroup$ +1 I think this is a good question not a stupid one. $\endgroup$ Sep 18, 2012 at 11:01

2 Answers 2


If you run PCA on ${\bf Y}_i - \hat f({\bf X}_i)$, you are visualizing residuals $\epsilon_i$, not the original data ${\bf Y}_i$. In other words, you will analyze the conditional variance of ${\bf Y}_i$, and that may be of interest per se. If it is the variability of ${\bf Y}_i$ that you want to visualize, and you know these outcomes are affected by ${\bf X}_i$'s, then what you are describing appears a moderately sensible visualization, with the caveat that 5D graphs are difficult to read and interpret. You could also look into principal curves if the dependence on ${\bf X}_i$'s is heavily non-linear.

  • $\begingroup$ There's no conditional distribution in the regression model so I don't see why are you talking about some conditional variance. Moreover I don't want to use any model, my question is about what happens if there is such a model but we don't use it. Thanks for mentioning the principal curves, I didn't know this notion. $\endgroup$ Sep 19, 2012 at 5:02
  • $\begingroup$ Oops sorry I'm still sleeping ;) You mean that the regression model is a modeling of the conditional distribution of Y given the covariates, right. $\endgroup$ Sep 19, 2012 at 5:05

I'm not sure if I agree with your statement about better interpretation for PCA when applied to iid variables. Normality helps because for normal rv's, variance is the natural measure of dispersion, but the iid condition isn't really necessary. When you don't have normal random variables, you can try Hubert et al's Robust PCA for skewed data: Hubert et al.: "Robust PCA for skewed data and its outlier map"

  • $\begingroup$ If the data are i.i.d. multinormal then the principal components define the ellipsoid associated to the covariance matrix; that's why I said the interpretation is better. Thanks for Hubert & al's paper. $\endgroup$ Sep 19, 2012 at 4:57
  • $\begingroup$ IF the data are iid normals, then you get a sphere (the covariance matrix is $\sigma^2I$). With normals, you always get ellipsoids (in a smaller dimension perhaps, because of non-zero correlations.) $\endgroup$
    – user765195
    Sep 20, 2012 at 3:24
  • $\begingroup$ the data are always multivariate. With i.i.d. multinormal ${\cal N}({\boldsymbol \mu}, \Sigma)$ one gets the ellipsoid associated to $\Sigma$, which is a sphere when $\Sigma=\sigma^2 I$. This point is clear. $\endgroup$ Sep 20, 2012 at 5:11
  • $\begingroup$ You still get an ellipsoid with dependent, non-identically distributed normals. The only difference is that the ellipsoid might really live in a smaller dimension. $\endgroup$
    – user765195
    Sep 21, 2012 at 3:08
  • $\begingroup$ I don't see what ellipsoid you are talking about. $\endgroup$ Sep 21, 2012 at 5:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.