New answers tagged

10 votes

Do Statistical Binning Algorithms Exist?

The most rational and elegant solution, and best performing in terms of mean squared error of estimates, is to use a method that borrows information across groups: either penalized maximum likelihood ...
user avatar
2 votes

overfitting of random forest in r

How does the 70 % accuracy of your CV compare to the rF's oob estimate? The behaviour you observe is to be expected for random forests, see also my old answer: https://stats.stackexchange.com/a/...
user avatar
0 votes

Should the residuals of a machine learning regression model be i.i.d.?

to my opinion: as so as "Residuals in a statistical or machine learning model are the differences between observed and predicted values of data." - you can always plot these differencies ...
user avatar
  • 41
1 vote

overfitting of random forest in r

Random Forest Classifiers (RFC) with 100% training accuracy are not necessarily problematic. Make sure you are optimizing your hyperparameters on a separate validation set, this is especially ...
user avatar
0 votes

How to interpret Mean Decrease in Accuracy and Mean Decrease GINI in Random Forest models

Relative Mean Decrease Accuracy? "In either case, is the Mean Decrease in Accuracy the number or proportion of observations that are incorrectly classified by removing the feature (or values ...
user avatar
1 vote

Fine-tune random forest on time series

You could give a dev package I wrote up a shot. It just wraps LightGBM (boosted trees) with some time series functionality such as fourier basis functions, 'custom' linear basis functions, and ar ...
user avatar
  • 1,350
1 vote
Accepted

Fine-tune random forest on time series

You can always include lagged values of the target variable to account for autocorrelation. However, for a Boolean target, that will likely not add a lot of value. Also, much of the autoregressive ...
user avatar
4 votes

Improving a forest model by dropping features below a percent importance threshold?

In theory, you could take such an approach, but there's a number of caveats. Firstly, whether this is helpful for predictive performance will be rather case specific and it could certainly make your ...
user avatar
  • 23.4k
3 votes

Showing machine learning results are statistically irrelevant

Piggybacking on Tim's answer. You clearly already have trained a better model, so just show your colleagues its results. Here's a note, though: R2 score could prove to be an unreliable metric ...
user avatar
0 votes
Accepted

Random Forest math pure Investigation

Depending on your level, a good starting point might be the overview article Bian and Scornet (2016) A random forest guided tour. "The present article reviews the most recent theoretical and ...
32 votes
Accepted

Showing machine learning results are statistically irrelevant

You answered yourself: I made two additional models (mean and last sample) which often match or beat the RMSE of the RF and ANN models published in the paper. The mean model just takes the mean of ...
user avatar
  • 117k

Top 50 recent answers are included