New answers tagged

0

You are evaluating model performance using the OOB misclassification error, which is sensitive to accuracy of classifications only. However, the trees in the forest are constructed by optimizing the Gini index, which is sensitive to the accuracy of predicted probabilities. Your results suggest that several pairs of variables can be used to obtain the same ...


1

Can I use the distribution in a leaf node to perform anomaly detection? Yes, in principle you can, but it might not be a good idea to use terminal-node-specific distributions to define outliers. Selected splits can be unstable, and thereby the minima, maxima, 1st and 3rd quartiles of the distributions in the terminal node, too. Assuming that a bigger value ...


0

As to how two apparently similar methods give such different results: This is due to how the predictions are generated. For a new observation, in a classification RF, each tree's prediction is a class label. The final RF prediction will take a majority vote over these predictions. This works well for for classification, but the proportion of trees that ...


1

Fit a random forest for regression. I.e., make the response variable a 0-1 coded variable. Then the predicted values in the terminal nodes will be means ranging between 0 and 1 (corresponding to proportions). The final prediction will be the average of these proportions (not a majority vote). See R example below: RF1 is a classification forest, where ...


0

It seems analogous to the computation of an effect size. For each tree, we get a difference between two MSE values. Averaging over trees gives the mean difference between the two MSE values. The standard deviation of the differences reflects the variation around the mean, a measure of residual error (cf. pooled standard deviation in ANOVA). Dividing the mean ...


0

If you are referring to the two importance measures returned by the randomForest package and function in R: mean decrease in accuracy reflects a permutation importance, computed using OOB error mean decrease in Gini impurity reflects the amount of reduction in the Gini index, that can be attributed to splits involving that variable


3

Permutation importances would be suited for your goal, you can compute them as follows (example R code apended below): Fit a RF on a training dataset. Compute MSE on test observations. For each predictor variable, randomly permute the values of that variable in the test set and compute the MSE again. The difference between this MSE and the MSE computed ...


2

This would only work for generalized linear models (GLMs). With 'work' I mean that you get perfectly identical error estimates (e.g., train and test MSE) using the original predictors, and using a single predictor, computed by weighing each of the coefficients by their respective regression coefficients. Random forests are not linear models. Any deviations ...


0

A simple example for a case when logistic regression can’t work properly: https://towardsdatascience.com/when-logistic-regression-simply-doesnt-work-8cd8f2f9d997


1

In the scenario you outline in the post and in comments, the predictions are atomic. Predictions for the feature vector of time $t$ don't depend on the feature vectors for any other times. This is because the time-series nature of the data is not reflected in any part of this model; the data are essentially a tabular. All we need to do to verify this claim ...


1

Unless I'm mistaken, the units of the Mean Squared Errors of your imputations are expressed in your variables units-squared, not in percentage. Therefore, I believe it is yours to interpret whether a Root Mean Square Error of 0.02, 0.007 or 0.017 is acceptable or not with regards to the units of your variables (as I'm not a statistician, I would be glad if ...


0

Random Forest and XGBoost don't select features. They inherently subsample the features at each tree/boosting iteration, which is part of their procedure to fight with overfitting. So, XGBoost doesn't produce a winning three with, say, 6 features. The feature selector algorithm can use any estimator, and it selects the features greedily (either forward or ...


0

It is a decent question, asked a decade ago. The chat is long gone. Question Given a random forest: That has bias in the outputs That is decently trained with decent parameters and is otherwise non-pathological has some similarity to this (link) Find: Elastic net regression, or other regularized regression, to calibrate the output Logistic regression ...


1

In the Cox proportional hazards model, you can make a statistical statement about which variables have statistically significant impact (coefficients that are different from zero). In the random forest model, you can say that those three variables are the most important according to the importance heuristic, but you can't make a statistical statement about ...


1

I think one of the differences is that missForest is, at least in its original form, a method for single imputation, i.e. imputing a single best imputation. It tries to find the "best" predictions for the missing values given some set of predictors that it identifies (e.g. using internal variable selection). It therefore does not account for ...


0

The cause of the nan score values was including a value of 1 as an option for min_samples_split. Although it is not explicitly stated in the documentation that this parameter cannot be 1, it makes sense when one stops to think about what this parameter means; one cannot split a node into subgroups if there is only 1 sample! This also explains why 25% of the ...


1

Generally if your training error is much lower than the test error, it indeed suggests overfitting. However, the training error would almost always be smaller than the test error, and Random Forest generally doesn't overfit (as long as the bootstrap samples and mtry ratio are good- rule of thumb- 2/3, and sqrt(# variables) respectively). You could try ...


1

There's already many good points by others e.g. about sparse feature spaces, when we know a step-function will not work well/when you don't have a good representation of the data (some of these may of course be a matter of creating better features first), as well as non-tabular data types where other approaches are known to work better especially for how to ...


2

Random forest fits multi-dimensional staircase to your data. It produces sharp edges in predictions. If your data are of continuous nature, then probably there are better methods to fit them. That doesn't mean however that "you should avoid random forest to fit them" :-) I don't think there is any kind of data where you "should be worried"...


0

Random Forests are prone to exhausing memory and causing out-of-memory errors (as compared to an incremental / batch learning method that fixes memory usage.


2

No, you do not need to retrain. Because the trees are built independently, removing one tree is (on average) the same as training one fewer tree. See also Sycorax's link in comments and this datascience.SE question. If you're really wanting to compare different numbers of trees, it would probably be more robust to train more than one forest for each number ...


2

That's certainly what you should do for better model comparison. In sklearn, all methods that have cv as its input, you can either input a CVSplitter object or an iterable containing training and test indices for each fold which can be obtained via split method of the KFold object. This way, you'll be using the same folds.


Top 50 recent answers are included