What is the relationship between differential analysis and hierarchial clustering?

Question

I'm currently in an internship for R bioinformatics, where I'm writing software for single-cell RNA sequencing analysis. We're looking for differentially expressed genes between groups, but I don't understand the current process on exactly how to do this.

Right now, I have my current data: I have over 20,000 samples across about 850 genes, and I don't have any pre-determined or attached grouping data to go along with it.

I don't understand exactly how to begin the analysis or what the process might look like. I do have some useful packages like DESeq2 and NMF to help me, but I'm having a hard time understanding the relationship between differential analysis and hierarchical clustering.

From what I understand, differential analysis focuses on finding differences between groups... However, my data lacks all grouping information; I've been told to use NMF, but that causes me problems with memory usage and such, and I don't understand exactly how it works...

Hierarchical clustering sounds similar to differential analysis in terms of how it categorizes things, but I was told it was not very robust and trustworthy.

I feel like I need to ask a deeper question, but I don't know much about statistics, period. I'm just a simple freshmen/sophomore in college, and I'm clueless. Can anyone give me a hand?

Answer 1

My understanding of non-negative matrix factorization (NMF, always spell out your acronyms) is that it produces groups for you. When you approximate a matrix V by W times H, the H matrix contains information about which clusters each of the columns of V contributes to. From the Wikipedia page for NMF: "If H_kj > 0, that fact indicates input data v_j belongs/assigned to kth cluster". They also mention that NMF is roughly equivalent not to hierarchical clustering, but to K-means clustering (https://en.wikipedia.org/wiki/K-means_clustering). The key to hierarchical clustering is the hierarchy: elements belong to groups, which belong to larger groups, and so on. Here, you just want to divide the dataset into a manageable number of groups to look at. NMF is just one way to cluster the data.

Without knowing more about your biology work, I would suggest the following. If you can grab a memory-manageable dataset, use just that -- maybe 1,000 samples on 10 genes -- to get used to the techniques. Perform NMF, read up on what it means, and then dive into your full dataset. Someone around here should be able to point you to big-data algorithms for NMF, but I'm not the right person for that...