I am self-studying Kevin Murphy's book Machine learning - A probabilistic perspective and stumbled upon the following paragraph on biclustering. I understand the independence assumption - but am not yet seeing how this allows one to cluster. Can someone provide the intuition behind this?
If I understand your question, the link between independence and clustering is this. That data points within each cluster are generated IID from a distribution belonging to the cluster.
Suppose we had a simple model, where each data point is assigned a colour token (red or blue) and a shape token (square or circle) and its colour-shape combination determines which probability distribution it is drawn from:
- red squares are IID Normal(0,1)
- blue squares are IID Normal(10,1)
- red circles are IID Normal(0,3)
- blue circles are IID Normal(10,3)
But when you get the data you generally aren't told which data points received which tokens. Or even how many different shapes / colours were involved. So if each combination is equally likely the resulting histogram of draws would look something like:
The inference process tries to recover this. The inference process he suggests isn't clear to me from the level of description given.