Given a matrix, each row with a list of numbers, find the most variable rows with an "interesting" pattern I have a file that can be considered as a matrix with each row representing a measurement of gene expression of different samples (each column is a sample). I want to find those genes with the most interesting patterns of expression across samples. That means that several samples have a similar expression range, another few samples have another range or expression. Basically I'm looking for variability among expression values but I hope to find not just random variation but meaningful patterns that can be used to separate my samples into biologically meaningful groups. 
I came up with this naive idea of using standard deviation at each row/gene and find those with highest standard deviation. But this doesn't seem to be a good method at all.
I also thought about taking the 75th and 25th quantile value and do a simple subtraction and report those with the highest difference, maybe top 20% of genes.  
I'm struggling with a statistically meaningful method to do this. I kind of feel this problem can be found in different context and there might be some tools/methods made to address it. Does anyone here have a suggestion or comment on the methods I mentioned?
 A: I would suggest princpal component analysis (or the closely related principal coordinates analysis, PCO).
PCA decovnvolves a high dimensional dataset (where here each gene can be thought of as a dimension) into a smaller number of dimensions, while still retaining as much of the variance as possible from the origianl data set. The first principal component of a dataset will explain the largest fraction of the variance. You can then look at the "loadings" of that component to look at what genes are contributing to it. The genes with the highest absolute loadings will be those with the pattern that distinguish best between samples. You would do the PCA in R with:
pca <- prcomp(expression_matrix)
weightings <- pca$rotations[,1]

Remember to put your genes along the columns and your samples down the rows. weightings will now contain the weightings for the first princple component, and selecting the entries in weightings with the largest absolute values will tell you which genes are contributing most. 
An alternative would be to do a bi-clustering of the data, and selecting those genes that look like the drive the clustering of the samples. 
