Lets assume mat_pages[] contains pages in the columns (which you want to cluster) and individuals in the rows. You can cluster pages based on individual data in Rby using the following command:
pc <- prcomp(x=mat_pages,center=TRUE,scale=TRUE)
The loadings matrix is the matrix of eigenvectors of the SVD decomposition of
the data. They give the relative weight of each PAGE in the calculation of scores.
Loadings with larger absolute values have more influence in determining the score
of the corresponding principle component.
However, I should also point out the short coming of using PCA to cluster pages. The reason for this is that loadings give larger weights to the PAGES with higher variation, regardless of whether this variation is actually because of the PAGE content or some other reason (may be technical or individual variation). The loadings do not necessarily reflect the true differences between groups, which (maybe) your main interest. BUT, this clustering truly reflects the differences in the group under the assumption that all the pages have same variance (I don't know if this is a valid assumption).
If you have a powerful computing facilities (which may be possible given your data size) - using hierarchical models may be a good idea. In R, it can be done using lme4 package.
What do do after you have the scores?
This is a crude suggestion and the analysis depends greatly on how the data looks like. Also, I would guess this process would be highly infeasible to group the data of magnitude that you have.
pc.col <- paste("page", 1:27000, sep=".")
pdf("principle-components.pdf")
plot(pc$x[,1:2]) ## Just look at the 1st two loadings (as you can see the groupings in a plane)
dev.off()
Hopefully, this can give you a picture of how the data is grouped into.
Warning: this is not what I would recommend.
My recommendation:
Problems like these arise frequently in genomics.In your case pages corresponds to genes and individuals corresponds to patients (basically individuals is has the same meaning as in genomics)
You want to cluster the pages based on data.
You can use a lot of clustering packages in R and have been pointed in other answers. A fundamental problem with packages is like hclust is how to determine the number of clusters. A few of my favorite ones are:
- pvclust (Gives you clusters and also gives a p-value for each cluster. Using the p-value you can determine the statistically significant clusters. Problem: requires a lot of computational power and I am not sure if it will work with data of your size)
- hopach (Gives you the estimated number of clusters, and the clusters)
- there are other packages available in Bioconductor, please check them out in the task view.
You can also use clustering algos like k-means etc. I am sure I saw a thread in this forum about clustering. The answers were very detailed. It was asked by Tal Galili if I remember correctly.