use cross validation to determine the cut off level

Cross validation http://en.wikipedia.org/wiki/Cross-validation_(statistics)

is a standard method to answer these sort of questions ( how many inputs to use/ which model to use). Basically you are trying to balance making a complex model that fits your training data well with making a simpler model that is less variable and so may generalise better to new data

eg k-fold crossvalidation.
Take your training data and randomly partition the data into (say k=10) equal sized groups $g_i$. Now consider you want to try cutoffs of {10,100,1000,10000}. Use one group $g_j$ of data for testing and the pooled other 9 for training/estimation. Now calculate the test error for each model and for each of the 10 'test sets' ( typically called validation sets). then for each of the cutoffs, estimate the average test error (over the 10 groups). Choose the cut off with the best performance on the average of the 10 'test' sets. Using this best cut off reestimate the model using all the training data.