# What is meant by "random vectors" in Random forest ?

In the beginning of https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf, it is explained that:

The common element in all of these procedures is that for the kth tree, a
random vector Θk is generated, independent of the past random vectors
Θ1, ... ,Θk−1 but with the same distribution;


One way that I understand random forests is, you have some data say of dimensions $N x P$, so now you grow some $Q$ amount of decisions trees which will be used as classifiers, and each tree will get a random subset of the data. However, in the paper there are two inputs to the trees, the data $x$ and a random vector. What is this random vector? Is it just a column array of randomly generated numbers, that adheres to the constraints of independence from the other random vectors ?

Breiman says the following:

The common element in all of these procedures is that for the kth tree, a random vector Θk is generated, independent of the past random vectors Θ1, ... ,Θk−1 but with the same distribution; and a tree is grown using the training set and Θk , resulting in a classifier h(x,Θk ) where x is an input vector.

In all the algorithms Breiman is talking about, every time we fit a single tree, we first make some random decisions about how that tree will be fit

• In bagging, we decide to fit using only some randomly chosen data points.
• In the random split selection example, we, at each split, randomly choose one of the top $k$ candidate splits.

Breiman introduces the notation $\Theta_k$ for all the random choices we make when fitting the k'th tree. This makes the tree a function of both the data and the random choices, hence the notation

$$h(x, \Theta_k)$$

Random forest goes like

• For each tree, we first decide to fit using only some randomly chosen data points, then at each split, we choose to only consider a random selection of possible variables.

What you describe here

you have some data say of dimensions NxP, so now you grow some Q amount of decisions trees which will be used as classifiers, and each tree will get a random subset of the data.

is called bagging. Random forest combines bagging with withholding random predictors from tree splits.

• Hi @MatthewDrury, hope you're doing well. If you don't mind, and if I understood well: 1) Does $\Theta_k$ here stand for the bootstrapping plus the random selection of variables/features (or columns) that are fed to each tree in the random forest algorithm? 2) Is $x$ here the entirety of the sample (an $n\times m$ matrix, where $n$ is number of rows and $m$ is the number of columns) or just the bootstrapped sample fed to each tree? Thanks in advance. Commented Apr 30, 2020 at 5:10