I have a data on methylated gene expression which follows a bimodal distribution. Can I use a multivariate outlier analysis to detect differentially methylated genes? What are the various methods of doing this and what assumptions come with them? I know similar analyses have been used in analyzing gene expression data, but they claim the data are assumed to follow a multivariate normal distribution.
No, you ought not assume that.
Finding outliers in any data set is tricky; assumptions are dangerous. Even if the data ought to come from a particular distribution, outliers change the parameters of that distribution.
If you know the form your data ought to take (that is, not just that it is multivariate and bimoodal, but the parameters associated with the distribution) you could simulate the data and see how often you get values as extreme as the ones you think are outliers.
But I think maybe you don't need outlier detection so much as some form of regression.
It is good to explore the shape of the data when plotted using a histogram. It's also good that you've observed these data roughly follow a bimodal distribution. It makes sense to want to report the size or effect of the difference in the data you've observed. This is a kind of exploratory analysis.
In the machine learning world, I would ask whether this is a supervised or unsupervised learning problem. In the supervised setting, we would have labels attached to the two types of genes here. If you were, say, measuring zinc finger in two ethnic groups, you could have a hypothesis of whether there is a difference between them. That is a supervised setting because you've controlled the groups to which you will draw comparisons explicitly. The simple T-test is effective for testing for differences in groups, given sufficiently large sample size, and is very robust to distributional assumptions.
In the unsupervised setting, you might not know whether there is a genetic marker for differences in zinc finger, but you ask "Are these data consistent with two (or more) distinct (and unknown) groups have statistically significant zinc finger expression?" And this becomes an unsupervised clustering problem. In this setting you may use the EM algorithm to iteratively maximize likelihood by estimating both the group means and group assignment in the data. A ML test against the null hypothesis (unimodal normal ML) is an effective test of hypothesis. This does introduce important distributional assumptions because this is not a test of means but a test of variances (which in general have less power and can be biased).
In either situation, neither of these methods can be called outlier detection. Because you're interested in significant and reproducible results which an outlier is not.