How to visualize/summarize a matrix with number of rows $\gg$ number of columns?

Question

I have a matrix of positive real numbers between 0 and 1; the rows represent genes and columns represent samples. Number of rows is greater than the number of columns by a magnitude of $10^4$. I am wondering how to visualize this in R. I know heatmap is one of the ways to do this, but are there other ideas. Here are a few points which I want to emphasize in the visualization:

Data:

1. The rows and columns have no order to them (as you may have already realized); specifically rows and columns are exchangeable.
2. The entries of the matrix are positive real numbers between 0 and 1.
3. A small fraction of the data (10% of the rows or genes, around 1000) are actually "interesting".
4. The matrix represents the estimated probability of genes being more active in a sample.

Aim:

1. I want show: which genes are more active and in which sample. The matrix has a lot of rows in which the probabilities are very similar across columns.
2. I am ok with ordering the rows (genes) to make the pattern clearer.

My thoughts:

At the moment I can determine active genes in a sample by choosing a cutoff (say $\ge 95\%$) and arrange the genes in such a way that first set of rows are active genes in sample 1, second set of rows are active genes in sample 2, ...

I was also thinking about visualizing a subset of the data, may be by sampling rows. But I did not have any success.

I know these ideas may not be very elegant but rearranges my data in a way which makes the pattern more recognizable.

I know similar questions have been asked before, but I thought my query was a bit more specific, so hopefully I can get better inputs from the members of this forum.

Answer 1

Find one-dimensional multidimensional scaling solutions for the rows and for the columns (separately), using whatever similarity measures you like (such as correlation). Sort the rows and columns according to their MDS positions. This will bring similar genes together and similar samples together. The whole thing can then easily be visualized as an array plot (e.g., normalize the values to the range 0..255 and display it as a grayscale image).

A 50 by 6 array of standard normal variates was processed in this way (using Euclidean distances as the proximity measures):

There's not much to see--after all, these data are iid--but look at the correlation matrices of the reordered columns and rows:

(red = positive, blue = negative). The concentrations of positive correlations along the diagonals and negative correlation off the diagonals demonstrate the method has worked as advertised. (With the original data, the correlation matrices are random, too, causing the red and blue cells to be more evenly interspersed throughout.)

In my experience, when there are even subtle nonzero correlations among rows and/or columns, this method does an excellent job of bringing them out in the original array plot (grayscale, above) and providing a visual display of clustering along both dimensions. Larger blocks along the diagonals of the corresponding correlation matrix plots help identify strongly clustered groups of rows or columns.

Answer 2

I was about to suggest something along @whuber's answer (I used this reordering technique but in a context of feature selection, so I was mainly concerned with the "variables view"). So let me suggest two other directions (but the first one is close to the already proposed one).

As far as heatmaps are concerned, you can display them after a slight rearrangement of rows (samples) and/or columns (genes) through hierarchical clustering (yet another aggregation method based on a (dis)similarity measure). There're a lot of R packages that can do this, for example the cim() function in mixOmics. Another package that might be of interest is MADE4; it relies on the very good ade4 package for multivariate data analysis and visualization.

If you are concerned with the rather large number of variables, you might also consider some reduction method for genes clustering. One that I've heard about is PCA-gene shaving (Hastie et al., 2000), that is largely described in Izenman (2008). In essence, this is a two-stage iterative procedure where (a) for feature selection, we single out genes whose correlation with the first principal component is below a distribution-based threshold (say, the 10% of genes having the lowest correlation at each step), and (b) for clustering, we seek to maximize an $R^2$ measure (between-cluster/within-cluster variances) for $j$ successive clusters of size $k_j$, where the optimal $k_j$ is obtained by a permutation technique and the use of the gap statistic (after effects of the former cluster has been removed by residualization). More detailed informations can be found in the paper referenced below, but the general idea is to cluster genes into small and potentially overlapping subsets of correlated genes that vary as much as possible across individuals.

References

1. Hastie, T., Tibshirani, R., Eisen, M.B., Alzadeh, A., Levy, R., Staudt, L., Chan, W.C., Botstein, D., and Brown, P.O. (2000). 'Gene shaving' as a method for identifying distinct sets of genes with similar expression patterns. Genome Biology, 1(2).
2. Izenman, A.J. (2008). Modern Multivariate Statistical Techniques. Springer.