1
$\begingroup$

I tried to understand it, by applying an regression-tree on the Friedman#1 dataset:

library("mlbench") 

dataset <- list()
random.forest <- list() 
regression.tree <- list() 
train.data.in.test.leaf <- list()
test.data.in.test.leaf <- list() 

test.leaf.size <- 0 
test.leaf.prediction <- 0 
test.leaf.index <- 1 
all.leaf.index <- matrix(1, nrow = 500, ncol = 1) 

dataset <- data.frame(mlbench.friedman1(501, sd=1)) 

Then I fitted a regression tree on the 500 first data-points in order to predict the 501-st data-point:

library("randomForest") 

random.forest <- randomForest(y ~ ., data=dataset[1:500,], ntree=1, nodesize=50, mtry=3)

regression.tree <- data.frame(getTree(random.forest)) 
status <- regression.tree$status; 
split.var <- regression.tree$split.var;  
split.point <- regression.tree$split.point; 


while(status[test.leaf.index] < (-1)) {
    if(dataset[501,split.var[test.leaf.index]] <= split.point[test.leaf.index]) { 
        test.leaf.index <- regression.tree$left.daughter[test.leaf.index] 
    } else { 
        test.leaf.index <- regression.tree$right.daughter[test.leaf.index] 
    } 
} 

for(i in 1:500) 
  while(status[all.leaf.index[i]] < (-1)) {
  if(dataset[i,split.var[all.leaf.index[i]]] <= split.point[all.leaf.index[i]]) { 
    all.leaf.index[i] <- regression.tree$left.daughter[all.leaf.index[i]] 
  } else { 
    all.leaf.index[i] <- regression.tree$right.daughter[all.leaf.index[i]] 
  } 
} 
end 

train.data.in.test.leaf <- dataset[all.leaf.index == test.leaf.index,]; 
test.data.in.test.leaf <- dataset[501,]; 

test.leaf.prediction <- regression.tree$prediction[test.leaf.index] 
test.leaf.prediction2 <- mean(train.data.in.test.leaf$y)

end 

test.leaf.index and all.leaf.index are important. They give us the leaf where the test-point lies and the leaves, where all other points lie. test.leaf.index and all.leaf.index are valid. I checked them with the function:

get_path_to_node <- function(tree, child){
  parent <- which(tree[,'left.daughter']==child | tree[,'right.daughter']==child)
  if( parent==1 ) return(paste(parent, child, sep='->'))
  return( paste(get_path_to_node(tree, child=parent), child, sep='->' ) )
}


path <- get_path_to_node(regression.tree,test.leaf.index) 

and by checking the split rules and split points.

Now my question is: why are test.leaf.prediction (=calculated with R's randomForest package) and test.leaf.prediction2 (=calculated manually) different?

$\endgroup$
  • 1
    $\begingroup$ Be aware that questions about code &/or code check are generally off topic here. Your software neutral question about RF is fine, but the code-based version may not get an answer. $\endgroup$ – gung - Reinstate Monica Jun 21 '17 at 11:42
0
$\begingroup$

I finally found out how the RF-package works: The RF-package has a parameter which is called

randomForest(..., replace = ..., ...) 

which is set

= TRUE 

as default. If it is set to

= FALSE 

and if you also set

randomForest(..., sampsize = nrow('of your training-data'),...) 

a single regression tree calculates its constant prediction for leaf $ R_j $ just by $ ave(y^{(i)}: i\in N_j) $, where $ N_j = \{n \in \mathbb{N} : [(x^{(n)})^T,(y^{(n)})^T]^T \in training-data \ \land \ x^{(n)}\in R_j \} $.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.