What kind of clustering algorithm for clustering samples according to OTUs

Question

I have data in the following form:

OTU Sample1 Sample2 Sample3 ...
1   1       0       2
2   3       5       0
3   0       5       1
.   .       .       .
.   .       .       .

So samples are sequencing data. Then all the sequences are pooled together and OTUs (operational taxonimic units) are calculated by an algorithm.

Now I want to see which samples are close to each other by clustering using OTU information as a "fingerprint". So in the example above Sample1 has 1 sequence in OTU1, Sample2 has five sequences in OTU2 and so on.

What kind of clustering algorithm would you suggest (so far I understand hierarchical clustering and k-means, but I'd like to learn others if needed). I am not sure what fits here the best

I would like to work with R.

Some small side question: Does anyone know an algorithm called "Jacks knife" or so? If yes, where do I find info regarding this algo?

Answer 1

I would suggest you look into some ordination methods, such as non-metric multidimensional scaling (NMDS), (canonical) correspondense analysis, or (canonical) redundancy analysis. Ordination methods typically give you a view of the closeness of the samples that is possibly easier to interpret than a clustering (for instance, a biplot).

In addition, digging into some metrics is probably a good idea. There are several metrics that are often used for sequence data, such as Hellinger transformation or Unifrac distance. Distance measures allow you to decide how you would like to measure the closeness of the samples.

I don't know what software you have used, but for example mothur (and R, of course) offer the mentioned metrics, ordination methods and visualization capabilities (mothur less, R more).

Here are some resources, you might find useful:

1. http://ordination.okstate.edu/
2. http://ubio.bioinfo.cnio.es/Cursos/CEU_MDA07_practicals/Further%2520reading/Papers%2520on%2520ecological%2520statistics/Transformations%2520in%2520ordination%2520Legendre2001Oecologia.pdf
3. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2121141/
4. http://store.elsevier.com/product.jsp?isbn=9780444538680
5. http://adn.biol.umontreal.ca/~numericalecology/numecolR/

Number 2 on the list above is very good article offering an overview of the ordination methods, and it also offers some solutions to the problems you might encounter. Number 3 discusses the same ideas in relation to microbial ecology.

In addition to the ordination methods, you can use also clustering methods. NMDS is often used with hierarchical clustering, because they both use the same distance matrix, but create a different visualization of the same data. Also, have you considered a heatmap, that displays the count table you have with color coding, and hierarchical clusterin trees of the samples and OTUs on the sided (an example image on, e.g., http://www2.warwick.ac.uk/fac/sci/moac/people/students/peter_cock/r/heatmap/)?

For jackknifing, have you consulted the Wikipedia page (http://en.wikipedia.org/wiki/Resampling_(statistics)) that offers and overview of the randomization methods? It is similar to bootstrapping, and both are often used for statistical testing (e.g., permutation testing) in community ecology.