2
$\begingroup$

Let $X$ be a matrix of $n$ rows and $m$ columns. $n$ is the number of samples and $m$ is the number of gene expressions. Gene expressions are basically numerical continuous values.

Assume we have a correlation matrix of size $n \times n$ where index (i,j) is the Pearson correlation value between sample $i$ and sample $j$ with respect to the gene expression values that they represent.

Here are 4 ways that I thought of using to detect outliers and would like to know if any of them make sense.

1) Compute the global mean of the correlations, say $mu$. Then perform one-sample t-test between each sample's correlation vector and the global correlation mean. The samples having a p-value smaller than 0.05 might indicate they are outliers.

2) Compute the global correlation mean and compute the euclidean distance between each sample correlation vector with the global correlation mean. Those that are very far away might suggest that they are outliers.

3) Instead of using correlation, compute the global mean of the gene expression values. Then perform one-sample t-test between each sample's gene expression vector and the global mean. Like in (1), the samples with a p-value smaller than 0.05 might indicate they are outliers.

4) Instead of using correlation, compute the global mean of the gene expression values and then compute the euclidean distance between each sample gene expression vector with the global mean. Those that are very far away might suggest that they are outliers.

Do any of these methods make sense ? Does using the correlation values for detecting outliers make sense ?

Thank you!

$\endgroup$
  • $\begingroup$ None of them will reliably detect outliers. Can you explain why do you want to re-invent the wheel? FYI there are methods for finding outliers and they come with some guarantee attached. $\endgroup$ – user603 Feb 27 '15 at 8:58
  • $\begingroup$ Thanks! Can you suggest some reliable methods ? I am looking for something very intuitive. Cheers. $\endgroup$ – johnny Feb 27 '15 at 17:06
  • $\begingroup$ For high dimensional outlier detection, methods with positive breakdown and having rotation equivariance come with strong guarantees. The most popular one is ROBPCA it is implemented in Matlab and R (package rrcov) $\endgroup$ – user603 Feb 27 '15 at 17:11
  • $\begingroup$ Very cool! I will use your suggested method after trying out basic intuitive ideas for outlier detection. One small follow-up question though. Using quantiles as shown as the "top answer" here (stackoverflow.com/questions/12866189/…) are they reliable enough to detect very extreme outliers ? Perhaps it is similar to the methods i suggested above. If you could perhaps provide some justification as to why these methods aren't reliable, it would be helpful to my understanding! Thanks! $\endgroup$ – johnny Feb 27 '15 at 18:41
  • $\begingroup$ unfortunately no because it is very easy to show that (even extreme) multivariate outliers need not be outlying on any particular dimension. $\endgroup$ – user603 Feb 27 '15 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.