The parameters required for a Random Forest classifier are as follows:
- Depth, $d$
- No. of random features, $K$
- No. of trees, $I$
- Randomizer seed, $R$
Which of the above are hyperparameters and which are model parameters?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
First, note that
randomizer seed might be an argument in the implementation of Random Forest you're using, but it is not a parameter of the RF algorithm itself (it just sets the generator of pseudo-random numbers of the machine in a state so that the results are reproducible). Also, the abbreviations you're using ($I$, $K$, etc.) are not standard and seem to be specific to your implementation.
Anyhow, regardless of notation issues, the two main parameters of RF are the number of trees grown and the number of predictors randomly tried at each split.
What you call depth is sometimes found as the maximum node size, and controls the size of the trees that are grown. In the original implementation of RF, trees are grown to the maximum potential extent so that they reach the lowest possible bias. Then, variance is reduced by growing many trees and averaging them. The reason is that in RF, you can only decrease error by reducing the variance (where $error = bias + variance$), so the bias needs to be as low as possible in the first place. Therefore, in most cases, you don't really need to adjust this parameter (depth or max nodesize). Just make sure the default value allows the trees to grow as deep as possible.
Note that in some instances, it seems that slightly reducing the sizes of the trees can help in reducing overfitting (see the comments in that thread).
Finally, for machine learning algorithms such as RF, Boosting, etc., hyper-parameters and parameters are the same things (the proper name would be hyperparameters though). There is a slight semantic difference between the two when dealing with probability distributions. See for instance:
Also, somehow related:
Random Forests converge with growing number of trees, see Breiman, 2001, paper. So if you set you ntree very high (for small datasets (n<1000) 10000 should be enough) your results get more stable and the effect of the seed reduces. The number of trees should be set as high as possible (although in some special dataset constellations it could possibly reduce the accuracy, see here: Difference in randomForestSRC and randomForest package / increasing OOB-Error curve)