# Density-lines in PCA plot?

Short version: Someone wants me to draw "density-lines" onto a PCA plot for every point on the PCA plot. Which does not make sense in my understanding. PCA is transforming the high dimensional data to principal components. How am I supposed to add a "density" for every point on a plot showing the two most relevant principal components?

Longer version:

1. I have high-dimensional data. Gene expression data, to be precise (about n = 22000 dimensions). The data is from k biological samples, so every sample has values for the about 22'000 dimensions.
2. Now I calculate the principal components and show the two principal components which explain most of the variation. The k samples now fall on the 2D plane reflecting the two most relevant principal components as they are calculated for the respective sample.
3. My collaborator now wants "denisty-curves" on the PCA plot for every one of the k points. Which does not work for a PCA plot, right? We reduce the n-dimensional space to fewer dimensions and then project the data onto this (in my case) two principal components. So in my understanding density lines for the reduced data points do not make sense, do you agree?

Example -- Let's assume a PCA plot as in the first figure here: https://cran.r-project.org/web/packages/ggfortify/vignettes/plot_pca.html Would you say that some sort of density lines for every point reflecting the underlying data does make sense in this case?

• This sounds like a request for a density estimate, such as a KDE, rather than for portraying some kind of confidence regions.
– whuber
Commented Oct 21, 2021 at 16:33
• In what way would it not make sense?
– whuber
Commented Oct 22, 2021 at 12:34
• You have made it clear that this is a density estimate: but what would be the matter with that? Because you object that it "goes against the idea," could you indicate which "idea" you have in mind and how a density estimate would be inconsistent with that idea?
– whuber
Commented Oct 22, 2021 at 16:40
• a collaborator from hell Commented Oct 25, 2021 at 11:21
• The density contours still make sense--it's just the request to draw the contours through each point that is nonsensical (and practically useless).
– whuber
Commented Oct 25, 2021 at 13:22

The data points are, after PCA transformation, distributed on the plane of the principal component axes.

That is an interpretation for which you can plot the density (ie. the density of the probability for a point to have a certain score on the principal components).

There are quite some examples with a search on the internet https://www.google.ch/search?q=pca+density+plot&source=lnms&tbm=isch

And in relation to your example with the iris dataset

I am linking to examples on Google because currently I have no R available to make a graph (maybe I will make an example myselve later). Below is an example from Fisher's article on the Iris data set. It is not PCA but LDA, and also it is not density lines but histograms/distributions. The principle in the sense that it is a projection onto a space of lower dimensions (in this case 1 dimension) and a display of the density distribution on that projected space.

"The use of multiple measurements in taxonomic problems" Annals of Eugenics, Vol VII, Pt. II, op. 179-188, 136

### Marginal distribution

When you have multivariate data, where the multiple variables are distributed according to some joint distribution, then you can still think of the distribution of a single variable. This is also called a marginal distribution.

It is not different for PCA and a distribution for the principal component. You can view this as the shadow of a multidimensional joint distribution being projected onto the lower dimensional space. The distribution of the data in this lower dimensional space is like a marginal distribution.

See below an example of this projecting. The data is distributed in 3 dimensions (according to some density) but you can project the points into a lower space and following that think about the density in that lower space.

Images from this question: Interpreting PCA figures in layman terms

• Please consider the fact that the density has to be drawn for every data point on the PCA plot. This was not accurately described by me initially. Does your reply still hold in this case? Commented Oct 25, 2021 at 10:24
• @Michael is the problem about how to make density plots or about the issue with density plots in connection with PCA. How would you make a density plot if your data did not have 22000 dimensions, but instead only 2 and there was no PCA involved? Would there be still problems? Commented Oct 25, 2021 at 10:44
• You write "Which does not work for a PCA plot, right? ". Maybe I do not completely understand what you are doing, but could you explain it by saying how you see it work for a non-PCA plot? Commented Oct 25, 2021 at 10:53
• If I would have only 2 dimensions initially (instead of the 22'000) then a density plot would NOT make sense in my case. Commented Oct 25, 2021 at 11:13
• There is a subtle difference between the PCA situation and the marginal density analogy: the PCA subspace is determined by the data rather than specified independently. Thus, it's not quite correct to interpret the PCA of the points as being some kind of marginal empirical distribution. I believe the KDE could still be interpreted as an estimator of a marginal distribution, but we have to bear in mind that the KDE will have slightly more uncertainty than expected (due to the uncertainty in the estimated PCs).
– whuber
Commented Oct 25, 2021 at 13:21

It sounds like there are two problems: one of methodology and another of conflict resolution. My initial impression is say to just "split the baby". At the fundamental level, the collaborator is just asking for a simpler way to visualize the results of the PCA so what could be the harm in delivering a "prettier graph." At the same time, everyone needs to be well aware as you rightly point out no one should try to make any inferences using the resulting marginals.

To that end, one proposed way to demonstrate the issue to the collaborator is to run a sensitivity analysis on the data to illustrate how much potential there is for estimation error in the method they have requested. You can bootstrap several samples or add varying degrees of noise and inspect how different the plots are between the full dataset and the augmented ones.

Here is an article on unsupervised dimensionality reduction on gene expression data which found that PCA was relatively robust in this regard: Comparative study of unsupervised dimension reduction techniques for the visualization of microarray gene expression data. That said, in Figure 6 of their paper they found that with a very low number of principal components, the issue you described can be very impactful.