Timeline for How to deal with a systematic bias in the random forest model, and what are possible alternative modeling approaches?
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 8, 2015 at 8:11 | comment | added | Fabi_92 | Can I also use the OOB estimates for variable importance to reduce dimension? | |
| Oct 7, 2015 at 14:07 | comment | added | Soren Havelund Welling | I agree with George and DJohnson also. You may be able to tune your model with variable filtering, I would personally favor highest absolute spearman correlation or variable importance. You need to wrap the entire process in a e.g. 10-fold cross validation to assess over fitting. | |
| Oct 7, 2015 at 14:01 | comment | added | Soren Havelund Welling | Actually that RF predictions "flattens out" / "gets a smaller slope", when the model regression is not perfect, is a very sensible 'Baysian-kinda' property. Think of your mean response as your prior. The RF model will effectively incorporate the uncertainty in its predictions, such that these will move closer to the mean response. The same is true for classification. Only when the regressor or classifier is performing 100%, will the predictions not be effected by the prior. | |
| Oct 7, 2015 at 12:30 | comment | added | George | This is a common problem of random forests. It often happens that random forests have problems predicting the tails of the outcome's distribution. I've read that one way to improve upon this is to perform a linear regression on each node instead of averaging, though I wasn't able to find any implementation. I don't recall where I read this. If I find the source I will post it here. | |
| Oct 7, 2015 at 12:23 | history | edited | amoeba | CC BY-SA 3.0 |
removed pca tag, edited title
|
| Oct 7, 2015 at 12:18 | answer | added | user78229 | timeline score: 1 | |
| Oct 7, 2015 at 11:28 | review | First posts | |||
| Oct 7, 2015 at 11:33 | |||||
| Oct 7, 2015 at 11:26 | history | asked | Fabi_92 | CC BY-SA 3.0 |