I have an $n\times p$ matrix, where $p$ is the number of genes and $n$ is the number of patients. Anyone whose worked with such data knows that $p$ is always larger than $n$. Using feature selection I have gotten $p$ down to a more reasonable number, however $p$ is still greater than $n$.

I would like to compute the similarity of the patients based on their genetic profiles; I could use the euclidean distance, however Mahalanobis seems more appropriate as it accounts for the correlation among the variables. The problem (as noted in this post) is that Mahalanobis distance, specifically the covariance matrix, doesn't work when $n < p$. When I run Mahalanobis distance in R, the error I get is:

 Error in solve.default(cov, ...) :    system is computationally
 singular: reciprocal condition number = 2.81408e-21

So far to try solve this, I've used PCA and instead of using genes, I use components and this seems to allow me to compute the Mahalanobis distance; 5 components represent about 80% of the variance, so now $n > p$.

My questions are: Can I use PCA to meaningfully get the Mahalanobis distance between patients, or is it inappropriate? Are there alternative distance metrics that work when $n < p$ and there is also much correlation among the $n$ variables?

  • $\begingroup$ The PCA could break the variable correlations, unless you use something like an oblique rotation. I'm also not sure how the variance apportioning in PCA will affect the Mahalanobis distance between similar patients. $\endgroup$
    – Michelle
    Mar 7, 2012 at 4:48
  • $\begingroup$ If PCA breaks the variable correlations, could I use some other distance metric (such as Pearson's distance) instead of the Mahalanobis distance? $\endgroup$
    – user4673
    Mar 8, 2012 at 19:22
  • $\begingroup$ I don't know enough to recommend further. I was hoping someone else would jump into the conversation. :) Given how variable reduction techniques like PCA work, I am curious as to whether any distance metric can be used on the outputs. $\endgroup$
    – Michelle
    Mar 8, 2012 at 19:40
  • $\begingroup$ I'm not an expert in the area. I just want to share, that I've a smilar problem of yours, and I've used 'cov.shrink' from corpcor package. $\endgroup$ Jun 8, 2012 at 13:56

3 Answers 3


If you keep all the components from a PCA - then the Euclidean distances between patients in the new PCA-space will equal their Mahalanobis distances in the observed-variable space. If you'll skip some components, that will change a little, but anyway. Here I refer to to unit-variance PCA-components, not the kind whose variance is equal to eigenvalue (I am not sure about your PCA implementation).

I just mean, that if you want to evaluate Mahalanobis distance between the patients, you can apply PCA and evaluate Euclidean distance. Evaluating Mahalanobis distance after applying PCA seems something meaningless to me.

  • 1
    $\begingroup$ Here's a short article discussing the relation: Brereton, R. G. The Mahalanobis distance and its relationship to principal component scores Journal of Chemometrics, Wiley-Blackwell, 2015, 29, 143–145. dx.doi.org/10.1002/cem.2692. Note that PCA in chemometrics is by default a pure rotation of the data, thus not the unit-variance version Dmitry refers to. $\endgroup$ Aug 19, 2016 at 8:14

Take a look into the following paper:

Zuber, V., Silva, A. P. D., & Strimmer, K. (2012). A novel algorithm for simultaneous SNP selection in high-dimensional genome-wide association studies. BMC bioinformatics, 13(1), 284.

It exactly deals with your problem. The authors suppose the use of a new variable-importance measurements, besides that they earlier introduced a penalized estimation method for the correlation-matrix of explanatory variables which fits your problem. They also use the Mahalanobis distance for decorrelation!

The methods are included in the R-package 'care', available on CRAN


PCA scores (or PCA results) are used in the literature to calculate Mahalanobis distance between sample and a distribution of samples. For an example, see this article. Under the "Analysis methods" section, the authors state:

Data sets of fluorescence spectra (681) are reduced into a lower dimension (11) by evaluating the principal components (PCs) of the correlation matrix (681 × 681). PC scores are estimated by projecting the original data along the PCs. Classification among the data sets has been done using Mahalanobis distance model by computing Mahalanobis distances for the PC scores.

I have seen other examples of PCA/Mahalanobis distance based discriminant analysis in the literature and in the help menu of the GRAMS IQ chemometrics software. This combination makes sense since Mahalanobis distance does not work well when the number of variables is greater than the number of available samples, and PCA reduces the number of variables.

One-class classification machine learning algorithms (i.e. Isolation Forest, One-ClassSVM, etc.) are possible alternatives to the PCA/Mahalanobis distance based discriminant analysis. In our lab, Isolation Forest combined with data pre-processing has produced good results in the classification of Near Infrared spectra.

On a slightly related note, outlier or novelty detection with PCA/Mahalanobis distance, for high dimentional data, often requires calculation of the Mahalanobis distance cutoff. This article suggests that the cutoff can be calculated as the square root of the chi-squared distribution's critical value, assuming that the data is normally distributed. This critical value requires the number of degrees of freedom and the probability value associated with the data. The article appears to suggest that the number of principal components retained equals the number of degrees of freedom needed to calculate the critical value because the authors used the number of features in the data set for their calculation.

  • $\begingroup$ Welcome to our site. Because it isn't explicit in your answer, could you explain how it addresses the $n\lt p$ circumstance posited in the question? $\endgroup$
    – whuber
    May 23, 2019 at 17:13
  • $\begingroup$ When n < p, PCA is used to reduce p. The number of components retained from PCA analysis (nc) is usually much smaller than p, and it may also be smaller than n depending on the size of n. Thus, PCA changes the "n by p" problem to an "n by nc" problem. In our lab where we analyse NIR spectra, p represents the number of wavelengths, which is typically greater than two thousands. n represents the number of training samples (~ 20 to 150). nc is typically 3 to 11. When n > nc, PCA/Mahalanobis analysis produces good results. $\endgroup$ May 23, 2019 at 17:59
  • 1
    $\begingroup$ The issue isn't what PCA does. That's well covered in other threads on this site. The problem is that the mahalanobis distance typically cannot be computed in this case because of the singularity of the covariance matrix: this is the circumstance described in the question. $\endgroup$
    – whuber
    May 23, 2019 at 18:03
  • $\begingroup$ Thank you for your kind comments. It appears that we have different interpretations of what the user is asking. The accepted answer states, "if you want to evaluate Mahalanobis distance between the patients, you can apply PCA and evaluate Euclidean distance. Evaluating Mahalanobis distance after applying PCA seems something meaningless ...". I think the last sentence is at odds with the literature I have seen and cited. What do you think? Best regards. $\endgroup$ May 23, 2019 at 19:15
  • $\begingroup$ I wouldn't let a possible misinterpretation by another respondent--even when it's accepted!--get in the way of a careful analysis of the question as it was asked. This one is a set of related questions, all restated in the last paragraph of the original post. $\endgroup$
    – whuber
    May 23, 2019 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.