1
$\begingroup$

I am training a model using random forest in Python. The dataset is about 5 million entries with ~60 numerical features, and my target is to do a classification among 30 different groups.

When I apply the model to the data I used for training, it gives almost perfect prediction. That makes me feel like I am overfitting this.

In general, what is supposed to happen if one trains a random forest model with a dataset, then make predictions on the exactly same data? What are good and bad signs, and what shall one consider doing in any bad scenario?

$\endgroup$
  • 3
    $\begingroup$ You would do better with validation, so splitting your data into training and validation sets to see how well your method predicts data it has not used for training. The next step is cross-validation, doing this several times to tune your model or even choose your model. And even better if before that you has reserved part of your data set for testing before moving into cross-validation $\endgroup$ – Henry Aug 25 '17 at 22:43
2
$\begingroup$

It depends what you are classifying. It might just be a data-set which is very easy to classify since the classes are well separable. The only way to know for sure if you are overfitting is by comparing this training set performance to out-of-sample predictions.

$\endgroup$
0
$\begingroup$

You always want to prevent overfitting by checking the prediction on a test set, but where you know the results.

I suggest you try the following:

In Python, you can use the function train_test_split to split your data to train / test.

Then use GridSearchCV to tune your models parameters.

When you are done with tuning, fit the predictor to train data, predict on the test data and see the results. This should give you a hint how well your model will perform on new, unknown data.

You can check out my ipython notebook I made recently for a friend, where is coded how this can be done (+ some more stuff not relevant to the question, only the first 6 cells are relevant).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.