# When you do PCA (or any dimensionality reduction), what is "the number of dimensions"?

Fundamental question

When you do PCA (or any dimensionality reduction), what is "the number of dimensions"? I always thought that the thing you measure (ie, the variable) is the number of dimensions: eg, if you measure the length, width, height of a box, that's 3 dimensions (3 variables); if you measure the abundance of 10,000 genes in 200 cells, that's 10,000 dimensions (not 200 dimensions).

More specific question

In regard to image 1 (below), what is the "correct" interpretation of number of dimensions (before PCA); is it the number of cells (200), or the number of genes (10,000)?

Note: I think it's possible to use either the number of cells or the number of genes as the number of dimensions, with obviously different interpretations. Additionally, there are some other good discussions of PCA on Cross Validated; however, my question is a little bit different: I'm really hoping for a response regarding my confusion after watching this video on PCA. Here's a brief explanation of my confusion.

The narrator is trying to explain PCA in context of this experiment (Image 1, below):

This graph was drawn from single-cell RNA-seq. There were about 10,000 transcribed genes in each cell.

Each dot represents a single-cell and its transcription profile. The general idea is that cells with similar transcription should cluster.

As I thought I understood PCA, in this experiment, the genes are the "dimensions" and the cells are the observations; ie, if there are 10,000 genes, there are 10,000 dimensions. This understanding seems to coincide with a different example (see references) using the Iris data set (Image 2, below); as you can see, the number of dimensions is the number of features of the flowers that were measured.

However, in the video, the narrator goes on to describe the number of dimensions as the number of cells for which the experiment measured the gene abundance (see Images 3 and 4, below):

Q: In regard to the experiment in Image 1 where "each dot represents a single cell", was the number of dimensions (before PCA) the number of cells or the number of genes?

References:

For link to reference on the Iris data, google "Principal Component Analysis in 3 Simple Steps Sebastian Raschka" (I don't have enough reputation on this site to include more links in this question).

• Transpose your matrix. You can clearly do PCA on either. So while the usual approach would be samples are cells, there is nothing preventing you from considering genes to be samples, and cells to be dimensions. May 6, 2017 at 19:58

(I cannot watch the video right now so this answer is to some extent a guess of what is meant)

First of all, yes we mostly talk about variates spanning the dimensions. However, it is also possible to take the opposite view (this is sometimes calles R-mode vs. Q-mode analysis).

Let me take a detour to cluster analysis to illustrate this: Cluster analysis with variates = genes = dimensions will look for groups of cases that have similar gene expression patterns. On the other hand, you can also take a "transposed" view and ask for groups of genes which are expressed similarly for the same cells. Genes and cells have changed their role compared to the first approach. For some types of data, you may get similar groupings both ways (see e.g. our paper on using this for spectroscopic data: A. Bonifacio, C. Beleites and V. Sergo: Application of R-mode analysis to Raman maps: a different way of looking at vibrational hyperspectral data, AnalBioanalChem, 407, 4 (2015) 1089–1095. DOI 10.1007/s00216-014-8321-7) whereas for other types of data both ways of looking at the data are interesting in themselves (e.g. for genetic data). In the latter case, you can use a heatmap giving both ways of clustering.

Now for PCA, the fun fact is that up to some decisions of standardization (row vs. columns for centering and possibly scaling) you'll arrive at the same solution both ways - just scores and loadings will change their role. (see e.g. https://stats.stackexchange.com/a/147983/4598 and Why PCA of data by means of SVD of the data? for more details)

Is the number of dimension the number of cells or the number of genes?

IMHO this is rather ambiguious and as explained above depends on the view of the data you take (i.e. the question you ask/the application at hand).

For PCA, there's the additional ambiguity that "dimensions" is sometimes also used refering to the rank of the data matrix. The rank cannot be more than the smaller of number of rows and number of columns and it is also the maximum number of principal components for that data matrix and thus the number of dimensions of the resulting rotated coordinate system (before reducing dimensions by truncating this coordinate system).

In your example of 200 cells and 10⁴ genes the PC will at most span 200 dimensions, regardless of whether cells or genes were considered the variates by the mode of the data analysis.

• Thank you, I appreciate your answer. I'll read more about R-mode vs Q-mode analysis. If you have more time in the future, I am still looking for clarification regarding the explanation provided in the video. If I can make the question more clear, please let me know.
– John
May 6, 2017 at 15:43

An extensive discussion is already provided here in the answer by cbeleites, and under similar questions (PCA and Correspondence analysis in their relation to Biplot), so I'll just comment briefly on the specific video.

As the narrator never mentions "scores" or "loadings" explicitly throughout the video, and the term "dimensions" in PCA is already ambiguous, technically there were no mistakes. However, I agree with you that his presentation is confusing: the first part states that the dimensionality along cells is to be reduced (200 cells -> 2 PCs), and the second part actually focuses on reducing dimensionality along genes (10,000 genes -> 2 PCs). I'd say there's many better and still accessible introductions to PCA, with consistent presentation and actual terminology E.g.: http://webspace.ship.edu/pgmarr/Geo441/Lectures/Lec%2017%20-%20Principal%20Component%20Analysis.pdf .

• Thank you! That's what I was thinking, but I wasn't sure.
– John
May 6, 2017 at 17:40

The definition of a vector space is quite general and there are numerous ways to represent data as vectors in a vector space.

From my cursory examination, they may be doing the following?

• Let $i = 1, \ldots, m$ index the gene.
• Let $j = 1, \ldots, n$ index the cell.
• Let $x_{i,j}$ denote the gene expression level of gene $i$ in cell $j$.

We then have a matrix of data $X$. You can run PCA on either $X$ or the transpose $X^T$.

### Treat columns of X as vectors (what you're naturally thinking)

You're thinking we can construct a vector for each cell $j$ as:

$$\mathbf{y}_j = \begin{bmatrix} x_{1,j} \\ x_{2,j} \\ \ldots \\ x_{m,j} \end{bmatrix}$$

That is, each vector $\mathbf{y}_j$ shows the gene expression levels of cell $j$ (and each index is a different gene).

### Treat rows of X as vectors (what they appear to be doing in Image 3 and 4?)

We could also form vectors using the columns of $X$

$$\mathbf{z}_i = \begin{bmatrix} x_{i, 1} \\ x_{i, 2} \\ \dots \\ x_{i, n} \end{bmatrix}$$ That is, each vector $\mathbf{z}_i$ shows the gene expression levels of gene $i$ (and each cell is a different index of the vector).

## Onward to PCA

Once you have a bunch of vectors, you can always conduct PCA to find an alternative basis for that space.

A subject area specific interpretation of that basis of course will depend on what your various vectors represent.

• I like you're answer here, but I think your description of rows and columns was different from what I was thinking. I'm using the Iris data image as my conceptual model. Using that model, I would use cells as "instances" and make them the row labels; I would use genes as the "features" and and make them the column labels. So, then, the vector for each cell is a row vector showing the expression of each gene, and the vector for each gene is a column vector showing the expression in each cell.
– John
May 6, 2017 at 17:36
• @John Yeah, my point is simply that from a pure mathematics standpoint, you can just as easily make the genes "instances" and the cells "features." (I don't know if that makes any sense or not or what the interpretation would be. I'm not a biology/genetics person.) May 6, 2017 at 17:40
• I think I understand conceptually what you mean, that you can treat either axis as the 'instances' or 'features'. I'm trying to make this more concrete, to be sure that I'm understanding. So, again, using the Iris data, you measure the characteristics of different flowers. The flowers are the instances or objects while the petal characteristics are the features. Similarly, with the RNA seq example, cells would be instances or objects, and genes would be the characteristics that are being measured. I think I got it, thank you!
– John
May 6, 2017 at 17:53
• I switched "rows" and "columns" in the post to match these comments.
– John
May 6, 2017 at 17:56

I don't like these videos. They only make understanding PCA more difficult by bringing in irrelevant details. Also, they're long and wordy.

The idea of PCA is very simple when it comes to applications. You have a several series of data, call them variables. Say you have N variables (series) $x_1(t),x_2(t),\dots,x_N(t)$.

Sometimes there are a few underlying factors that drive all these series. Let's say there are M factors $f_1(t),f_2(t),\dots,f_M(t)$, and you suspect (or know for sure) that they drive the variables: $$x_1(t)=c_{11}f_1(t)+\dots+c_{1M}f_M(t)\\\dots\\ x_N(t)=c_{N1}f_1(t)+\dots+c_{NM}f_M(t)$$

So, you're interested to extract the factor values $f_j(t)$ and the coefficients $c_{ij}$. So, PCA is one way of accomplishing this. In fact, if you don't know what is the exact number of components M, it can help you find that out too.