I am trying to dummy code a variable sector which have 20 levels. I am trying to combine some of levels. However I got this situation, if I have a say 2000 observations, how do I decide on the minimum frequency of a particular level that I should consider before making it a dummy variable, as variable with low frequency not add much information. How do I decide on what minimum % of total number it should be, in order to take into modeling?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
Cross validation http://en.wikipedia.org/wiki/Cross-validation_(statistics)
is a standard method to answer these sort of questions ( how many inputs to use/ which model to use). Basically you are trying to balance making a complex model that fits your training data well with making a simpler model that is less variable and so may generalise better to new data
eg k-fold crossvalidation.
Take your training data and randomly partition the data into (say k=10) equal sized groups $g_i$. Now consider you want to try cutoffs of {10,100,1000,10000}.
Use one group $g_j$ of data for testing and the pooled other 9 for training/estimation. Now calculate the test error for each model and for each of the 10 'test sets' ( typically called validation sets).
then for each of the cutoffs, estimate the average test error (over the 10 groups). Choose the cut off with the best performance on the average of the 10 'test' sets. Using this best cut off reestimate the model using all the training data.
