3 grammar (?) edited Oct 27 '15 at 21:35 John 19.1k33 gold badges4242 silver badges8080 bronze badges Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter $$K$$, in all our works with DRF that follows the publication of this paper, we have always useused a completely random value, that is to say, a value randomly chosen between $$1$$ and $$M$$, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there isare a lot of irrelevant features in the dataset, the traditional value $$M^.5$$ is a very poor choice. In this case, selecting a value at random for each node of the trees allows us to overcome this phenomenom. In all other cases, $$M^.5$$ is a good choice. Regarding the weighting process, I admit it could have been better explained. AtAt each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance, the ratio of trees that have predicted the true class, but only considering trees for which the concerned instance is an out-of-bagbag; and (ii) replace the previous weights by the new ones, computed from this ratio. Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I can notcannot give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details. Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter $$K$$, in all our works with DRF that follows the publication of this paper, we have always use a completely random value, that is to say a value randomly chosen between $$1$$ and $$M$$, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there is a lot of irrelevant features in the dataset, the traditional value $$M^.5$$ is a very poor choice. In this case, selecting a value at random for each node of the trees allows to overcome this phenomenom. In all other cases, $$M^.5$$ is a good choice. Regarding the weighting process, I admit it could have been better explained. At each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance, the ratio of trees that have predicted the true class, but only considering trees for which the concerned instance is an out-of-bag and (ii) replace the previous weights by the new ones, computed from this ratio. Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I can not give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details. Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter $$K$$, in all our works with DRF that follows the publication of this paper, we have always used a completely random value, that is to say, a value randomly chosen between $$1$$ and $$M$$, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there are a lot of irrelevant features in the dataset, the traditional value $$M^.5$$ is a very poor choice. In this case, selecting a value at random for each node of the trees allows us to overcome this phenomenom. In all other cases $$M^.5$$ is a good choice. Regarding the weighting process, I admit it could have been better explained. At each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance the ratio of trees that have predicted the true class but only considering trees for which the concerned instance is an out-of-bag; and (ii) replace the previous weights by the new ones computed from this ratio. Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I cannot give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details. 2 add latex edited Oct 27 '15 at 21:03 Silverfish 15.7k1616 gold badges7070 silver badges151151 bronze badges Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter K$$K$$, in all our works with DRF that follows the publication of this paper, we have always use a completely random value, that is to say a value randomly chosen between 1$$1$$ and M$$M$$, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there is a lot of irrelevant features in the dataset, the traditional value M^.5$$M^.5$$ is a very poor choice. In this case, selecting a value at random for each node of the trees allows to overcome this phenomenom. In all other cases, M^.5$$M^.5$$ is a good choice. Regarding the weighting process, I admit it could have been better explained. At each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance, the ratio of trees that have predicted the true class, but only considering trees for which the concerned instance is an out-of-bag and (ii) replace the previous weights by the new ones, computed from this ratio. Then Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I can not give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details. Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter K, in all our works with DRF that follows the publication of this paper, we have always use a completely random value, that is to say a value randomly chosen between 1 and M, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there is a lot of irrelevant features in the dataset, the traditional value M^.5 is a very poor choice. In this case, selecting a value at random for each node of the trees allows to overcome this phenomenom. In all other cases, M^.5 is a good choice. Regarding the weighting process, I admit it could have been better explained. At each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance, the ratio of trees that have predicted the true class, but only considering trees for which the concerned instance is an out-of-bag and (ii) replace the previous weights by the new ones, computed from this ratio. Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I can not give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details. Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter $$K$$, in all our works with DRF that follows the publication of this paper, we have always use a completely random value, that is to say a value randomly chosen between $$1$$ and $$M$$, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there is a lot of irrelevant features in the dataset, the traditional value $$M^.5$$ is a very poor choice. In this case, selecting a value at random for each node of the trees allows to overcome this phenomenom. In all other cases, $$M^.5$$ is a good choice. Regarding the weighting process, I admit it could have been better explained. At each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance, the ratio of trees that have predicted the true class, but only considering trees for which the concerned instance is an out-of-bag and (ii) replace the previous weights by the new ones, computed from this ratio. Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I can not give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details. 1 answered Oct 27 '15 at 20:54 Simon Bernard 12611 silver badge33 bronze badges Even if it has been few months these questions have been asked, I can still give some answers... Regarding the hyperparameter K, in all our works with DRF that follows the publication of this paper, we have always use a completely random value, that is to say a value randomly chosen between 1 and M, with equal probabilities. It has proven to be efficient in a large majority of cases. The process proposed in the paper is based on a previous work that shows that when there is a lot of irrelevant features in the dataset, the traditional value M^.5 is a very poor choice. In this case, selecting a value at random for each node of the trees allows to overcome this phenomenom. In all other cases, M^.5 is a good choice. Regarding the weighting process, I admit it could have been better explained. At each step (before growing a new tree in the forest), the idea is: (i) to evaluate for each training instance, the ratio of trees that have predicted the true class, but only considering trees for which the concerned instance is an out-of-bag and (ii) replace the previous weights by the new ones, computed from this ratio. Then, when the weights have been computed, they are used in two parts of the learning algorithm: (i) in generating the bootstrap samples (a high weight means a high probability to be selected in the bootstrap sample use for the new tree) and (ii) in the computation of the gini index. For combining the tree predictions, we use a majority voting, as it is done in most of the random forest methods. Unfortunately, I can not give any stable implementation of this algorithm for now, but as soon as I have properly re-written it, I plan to publish it on my website. Feel free to contact me if you need further details.