I'm trying to apply the bag of visual words approach to make scene classification. I started to use k-means to generate my codebook, but rapidly discovered its limitations. From one codebook generation to another, I get fluctuations of 10% on the classification ratio (data dimension: 8, number of clusters: 50). In addition, k-means is slow (I'm working on Matlab).

So I did some research and found that I can do unsupervised learning with random forests (paper: "Unsupervised Learning With Random Forest Predictors" by T. Shi and S. Horvath). It looked nice, so I decided to give it a try.

What I did:

(1) I created a synthetic data set from my original data set. To generate the coordinates of my synthetic samples, I take the corresponding coordinate of one of my original samples, chosen randomly (like explained here:http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#unsup). I also tried the Addcl2 method like explained in the paper that I mentioned above.

(2) The original data is labeled 1, the synthetic data is labeled 2. Then I train a random forest (I use Matlab's TreeBagger class) on this 2 classes. Then I compute the proximity matrix, but only for the original data.

(3) I use this proximity matrix as a distance matrix to perform unsupervised learning with hierarchical clustering.

Here is what I get for 2D data: Example for well separated data Example for badly separated data

It works approximately well for good separated data points, while the clustering result depends from one time to another, sometimes I get the same result as k-means. For badly separated data, the result is awfull.

What am I doing wrong? I could need some advice ;-)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.