17
$\begingroup$

I came across the term proximity in random forests. But I couldn't understand what it does in random forests. How does it help for classification purposes?

$\endgroup$

2 Answers 2

18
$\begingroup$

The term "proximity" means the "closeness" or "nearness" between pairs of cases.

Proximities are calculated for each pair of cases/observations/sample points. If two cases occupy the same terminal node through one tree, their proximity is increased by one. At the end of the run of all trees, the proximities are normalized by dividing by the number of trees. Proximities are used in replacing missing data, locating outliers, and producing illuminating low-dimensional views of the data.

Proximities

The proximities originally formed a NxN matrix. After a tree is grown, put all of the data, both training and oob, down the tree. If cases k and n are in the same terminal node increase their proximity by one. At the end, normalize the proximities by dividing by the number of trees.

Users noted that with large data sets, they could not fit an NxN matrix into fast memory. A modification reduced the required memory size to NxT where T is the number of trees in the forest. To speed up the computation-intensive scaling and iterative missing value replacement, the user is given the option of retaining only the nrnn largest proximities to each case.

When a test set is present, the proximities of each case in the test set with each case in the training set can also be computed. The amount of additional computing is moderate.

quote: https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm

$\endgroup$
2
  • $\begingroup$ What does "nrnn" stand for? I was reading Adele Cutler's (or possibly Breiman's as I can't tell who's authoring what here) page on RFs, and I can't find where they define nrnn. (It may very well be a linear algebra term with which I happen to be unfamiliar. $\endgroup$ Oct 20, 2017 at 4:10
  • 1
    $\begingroup$ nrnn = the number of nearest neighbors for which to compute proximities. Source : math.usu.edu/adele/RandomForests/ENAR.pdf page 161 $\endgroup$
    – klumbard
    Feb 15, 2019 at 18:58
1
$\begingroup$

Note that the authors of Elements of Statistical Learning state that "Proximity plots for random forests often look very similar, irrespective of the data, which casts doubt on their utility. They tend to have a star shape, one arm per class, which is more pronounced the better the classification performance." (p 595)

However, I think these authors don't mention the ways that random forests deal with missing data so much (even though they mention missing data with trees earlier in the book); perhaps the authors just did not highlight this aspect of RFs as much, which makes sense considering the book is enormous and has a lot of information on a lot of machine learning topics/techniques. However, I don't think that having the plots give similar shapes for any RF and data set means anything negative about RFs in general. For instance, linear regression basically always looks the same, but it's worthwhile to know what points lie close to the line and which seems to be outliers from the perspective of linear regression. So...their comment about the utility of proximity plots doesn't make sense to me.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.