New answers tagged

0

I'd like to point out that Random Forest is not just a bagging technique. It's a bagging + random subset of the features. Definition on Wikipedia suggests that ...The above procedure describes the original bagging algorithm for trees. Random forests differ in only one way from this general scheme: they use a modified tree learning algorithm that selects, ...


1

How ntree affects Random Forests in general? Will we get more precise "OOB error" (more trees -> bigger possibility that OOB sample will be part of a test set for more trees -> hence, precise OOB error?)? This is simple: the more trees, the better, where you should use something like 100, 250, or more trees if you can afford it. How set.seed() ...


1

Unfortunately the implementation of random forests between Python and R are not always directly comparable. If anything, because the random forest algorithm inherently performs bagging and random selection of explanatory variables (i.e. it samples both the rows and the columns of our training set when training), if this resampling is not done in the same ...


0

What this plots shows is a pairwise comparison of the variable importance rank associated with each metric. In particular: The lower diagonal part of the lattice shows the scatter plot between the variable importance ranks induced each of the two metrics on each variable. The blue line shows a LOESS smoother estimate and the grey bands the confidence ...


0

A couple things could be going on. Based on the nature of the algorithm the seed may not control the randomization. Also, GEE and R have a different default for the bag fraction, or how much data is withheld when growing the forest. I believe GEE is set to 0.5 and R is like 0.63 or so. Additionally, GEE can be set to run in out-of-bag mode or not.


1

I will be assuming that by "better performance" you mean better CV/validation performance, and not train one. I want to invite you to think of what the effect of log-transforming the target variable is on single regression trees Regression trees make splits in a way that minimizes the MSE, which (considering that we predict the mean) means that they ...


0

Given you've already come across the Bayesian approaches (Bayesian Additive Regression Trees, or BART), why don't you use those? In R the relevant packages would be 'dbarts' or 'BART'. 'dbarts' has a companion package 'bartCause' which adds wrappers for getting relevant causal estimands out of the model after estimation, although I'm not sure how good it as ...


3

Tangentially, the marginal distribution (that is, the distribution obtained when plotting a histogram) of the outcome is irrelevant in regression since most regression methods make assumptions about the conditional distribution (that is, the distribution obtained when plotting the histogram of the outcome were I to only observe outcomes which have the same ...


1

Random forests are based on the concept of bootstrap aggregation (aka bagging). This is a theoretical foundation that shows that sampling with replacement and then building an ensemble reduces the variance of the forest without increasing the bias. The same theoretical property is not true if you sample without replacement, because sampling without a ...


0

General answer: I would not be using SMOTE for a ratio of 7:3. If you really want to balance your precision/recall, try to set some class weights - they are available in R's implementation of Random Forest. 'sampsize' works that you need to give him a vector in the form of c(300,300), where the order depends on the level of the factor variable you passed ...


1

Partial dependence doesn't tell you the full story. You are looking at the marginal effect of one predictor on the response variable. It's just like the marginal distribution doesn't tell you the full story of the joint distribution. The graphs that motivate individual conditional expectation here (https://blogs.sas.com/content/subconsciousmusings/2018/06/12/...


1

I find Thomas's answer very detailed, but I will nevertheless add a couple things to it: Since random forest models already select features, is it possible to gain much by such a method? You could have some gains from feature selection in cases of highly correlated features and when having many unimportant features. Many high correlated features might ...


0

Could the reason for mtry being able to exceed p be because some of the variables are categorical, and automatically one-hot-encoded into several dummy variables?


0

Random Forest is a classification/regression algorithm. It can be used as a "feature selection" method in the sense that -once it has been trained for classification- it provides some Feature Importances based on the information that was gained when making splits on each variable. So technically yes, you can train your Random Forest on the full data and ...


2

Since random forest models already select features, is it possible to gain much by such a method? Yes and no. By selecting only a subset of features, and creating synthetic variables, you can help/accelerate the convergence of trees. But not necessarily improve it, because synthetic variables, which are a combination of one or more variables and one or ...


0

I think it would be more relevant to use a dimensionality reduction step first and then uses the resulting variables as the new responses. Assuming one uses Principal Component Analysis (I mention it as it is the simplest and most widely used dimensionality reduction technique), this would be akin to do Principal Component Regression and then looking at the ...


0

I understand that this output tells me that A and D are important variables because they have high MeanDecreaseAccuracy values. However, D is the inverse of A (they are perfectly correlated) so why does D have a higher MeanDecreaseAccuracy value? In general, it is not a good idea to have two perfectly correlated variables in your model. In Random Forest ...


0

A few things... a) Why are you using logistic regression? Revenue is continuous and bounded below by 0, where as logistic regression is for binary outcomes. b) It sounds like you are interested more by inference than prediction, so random forest -- though good at prediction -- really isn't what you want here. My advice is to use something like a ...


0

I would guess not, because the hyperparameter ‘mtry’ still injects randomness into the features that are included during a particular run, not to mention the randomness in the cases selected during cross validation. If anything I would suspect it becomes ‘more deterministic’ as mtry approaches # of features.


Top 50 recent answers are included