# How to bootstrap non-negative matrix factorization results?

I have RNA-seq data from 9 samples and around 15,000 genes. I know that these 9 samples consist of varying proportions of two cell types, each with their own expression profile. I am using non-negative matrix factorization with a rank of 2 to determine the cell-type ratios in each sample. In other words, I use NMF to decompose my 9 x 15,000 matrix into a 2 x 15,000 matrix and a 2 x 9 matrix.

I want to obtain confidence intervals for my estimates of the 2 x 9 matrix. From what I understand about bootstrapping, I could do this by resampling from my 9 observations a bunch of times and keep track of each estimate for each observation.

My question is, since NMF doesn't have any notion of observations vs. features, could I also estimate the variance by resampling genes (features) rather than observations?

-
The NMF is not, in general, unique, so some or all of the parameters may be unidentifiable. Why should you, then, expect to get confidence intervals for them? – cardinal Jun 17 '12 at 18:12
It would be polite, especially to new users posting their very first question, for downvotes to provide some constructive feedback on how they think the question could be improved. – cardinal Jun 17 '12 at 23:42
@cardinal: I can use outside information to map the NMF basis vectors to cell types, preserving identifiability. [this plot] (imgur.com/zvjvn) shows my estimates for cell type 1 for the 9 samples obtained by repeating the NMF 100 times (I know this doesn't give me a real confidence interval, I was looking at how much variability NMF itself has). If I was having identifiability issues, I would expect much greater spreads as the results would randomly alternate between the proportion and 1-proportion. – Amit Deshwar Jun 19 '12 at 14:27
@AmitDeshwar: please consider selecting one of the answers or explaining why neither satisfies your question. – jrennie Aug 20 '12 at 12:54

It turns out the answer is yes.
I read Efron and Tibshirani's textbook on the bootstrap and they gave an example of bootstrapping PCA applied to standardized test data where a few students answered many questions. In this example they resampled questions (equivalent to genes in my example). Since NMF is a special case of the more general multinomial PCA, it seems to follow that this approach should also work with NMF.

-

NNMF is unique to within a permutation and a scaling, per Laurberg, 2007, 14th IEEE/SP Workshop on Statistical Signal Processing, August 2007. Related, Donoho, Stodden "When does NNMF give a correct decomposition into parts?", Stanford, 2003.

-
 From Laurberg: "Theorem 8 states that the NMF is unique if the row vectors in W are boundary close and the column vectors in H are sufﬁciently spread". AFAICT, NMF is not generally unique to within permutation/scaling. – jrennie Aug 15 '12 at 13:31

Executive summary: "yes".

IIUC, your observed data is 9 rows by 15000 columns, and you have considered sampling-with-replacement from the rows to generate 9x15000 for variance estimation. As you probably suspected, this has the problem that a large number of the rows are likely to be duplicates of each other in each sample.

IIUC, you are proposing instead sampling columns. This sounds reasonable to me. Since there are 15000 columns, you will expect relatively few sampled columns to have a duplicate. Since you are only estimating a 2-rank matrix, I would sample less than 15000 columns, say only 1000 columns, and I would do the 1000 column sampling without replacement.

-
 I don't understand the justification for sampling without replacement only 1000 columns, this goes against the whole idea of bootstrapping. – Amit Deshwar Aug 21 '12 at 15:59 From the perspective of the 2x9 matrix, you have 15k samples of 9-dimensional data. Since you are only learning two principle directions, there is not much difference between 1000 vs. 15000 "samples". Yes, sampling without replacement is technically not bootstrapping, but, practically speaking (considering your problem), these are very small differences. – jrennie Aug 24 '12 at 15:35