2
$\begingroup$

I am using some ensemble classifiers such as Random forest for a classification task. I have tried tuning the classifier parameters, but failed to prevent overfitting.

Apart from tuning the parameters , are there any techniques to prepare data in such a way so that it doesn't overfit?

For example, my intuition is that if we divide values of numerical feature into groups, it will give less details thereby tendency to over fit will decrease.

Am I thinking in right direction ?

Improve this question
$\endgroup$
4
  • $\begingroup$ Throwing away data as you suggest is a bad idea to prevent overfitting. The usual solutions are to (i) get more data, (ii) use simpler models or (iii) control the complexity of your models better, for instance via strong regularization. $\endgroup$
    – Marc Claesen
    Oct 8, 2015 at 12:21
  • $\begingroup$ This depends heavily on the data that you have! How many covariates do you have and how many observations do you have? If the covariates are heavily correlated or redundant, then this may cause problems. Also if you have categorical variables with a lot of factors it might be a good idea to aggregate them into fewer factors. I would not aggregate a numerical variable into a categorical variable, unless there is a very good reason for why to do it. $\endgroup$
    – Gumeo
    Oct 8, 2015 at 12:27
  • $\begingroup$ I have around 42000 samples and 30 features. You guessed it right, one of my categorical features has 56 values, but how do I reduce them ( all have almost equal frequencies ) . Most of the pairwise correlations are around 0.001 so I guess they are not correlated.. Is this right ? $\endgroup$
    – mach
    Oct 8, 2015 at 12:46
  • $\begingroup$ with all respect, there's a good chance you're misinterpreting your RF model. check this thread: stats.stackexchange.com/questions/162353/… $\endgroup$
    – Soren Havelund Welling
    Oct 12, 2015 at 22:46

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.