Timeline for RandomForest classification model with 100% accuracy is it real or something wrong?
Current License: CC BY-SA 3.0
15 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 27, 2015 at 6:30 | history | edited | Danica |
edited tags
|
|
| Oct 4, 2015 at 2:03 | comment | added | Marc Claesen | @user777 is right. For an example of nested cross-validation, please refer to optunity.readthedocs.org/en/latest/notebooks/notebooks/…. | |
| Aug 19, 2015 at 12:21 | answer | added | prudenko | timeline score: 4 | |
| Jun 6, 2015 at 19:09 | comment | added | Sycorax♦ | No, you haven't. You only "get" one out of sample estimate per CV layer. You've used that OOS estimate to select the model hyperparameters. You'll need another batch of out-of-sample data to estimate performance for the selected hyperparameters in any reasonable fashion. | |
| Jun 6, 2015 at 14:26 | comment | added | unk1102 | Hi @user777 I am already using random split to split training data and test data and I have ran model multiple times with 100% accuracy. Random split makes sure no bias of training data set no? so don't you think I tried the approach you mentioned in different way | |
| Jun 5, 2015 at 23:55 | comment | added | Sycorax♦ | It's perfectly relevant. Otherwise you're testing on training data, which will bias performance estimates upwards. | |
| Jun 5, 2015 at 19:50 | comment | added | unk1102 | Hi I have quality data sets of 10 thousands rows out of which I am using 8000 for training and 2000 for testing model | |
| Jun 5, 2015 at 19:28 | comment | added | user3684792 | I am not sure the above comment is very relevant. How big is your data set? | |
| Jun 5, 2015 at 18:25 | comment | added | Sycorax♦ | Just write your own function. | |
| Jun 5, 2015 at 18:24 | comment | added | unk1102 | Any pointers/links please with R code sample etc to do what you just explained. Though I am using Spark MLlib Java but I will understand R code. | |
| Jun 5, 2015 at 18:19 | comment | added | Sycorax♦ | It's just like regular CV, except it has 2 steps. CV the whole data set. Take all but 1 fold, and CV partition that data. Then use the inner data to CV select hyperparameters. Then train a model on all the inner data, and test the selected model on the holdout set. Repeat for all outer holdout sets. | |
| Jun 5, 2015 at 18:16 | comment | added | unk1102 | Hi thanks for the response how do we do nested cross validation I have 80 % training data and 20% test data I did testing and error calculation on 20% data using model I created. Sorry I am very new to machine learning so please bear with my basic questions. | |
| Jun 5, 2015 at 17:59 | comment | added | Sycorax♦ | Did you train the model with nested cross validation? Selecting hyperparameters at the inner CV and then evaluating out-of-sample performance at the outer CV step? | |
| Jun 5, 2015 at 17:37 | review | First posts | |||
| Jun 5, 2015 at 17:43 | |||||
| Jun 5, 2015 at 17:33 | history | asked | unk1102 | CC BY-SA 3.0 |