What are approaches to estimate how many samples are needed for an SVM classification task in features extracted from text?
If I have for example a set of 8 labels in a 100-dimensional featurespace. How many samples do I need to build a reasonable classifier?
There is no such formula which gives you an exact number of required samples.
However you can incrementally verify whether adding more training data will be likely to yield better results.
Using learning curves you plot subsets of your training data (in increasing sizes) against the crossvalidation score (training score is usually included as well, but i am referring to the crossvalidation here). What i am referring to as score here, can be any evaluation metric, for example accuracy.
With a sufficient number of documents you will see that the slope your crossvalidation score curve will decline until you see only little improvements.
In the end it comes down to a cost-benefit considerations (assuming the training data limit has not been reached yet). For example for the left curve (Naive bayes, left one) here a cost-efficient amount of documents to use would be around n = 500.
For more information see: learning_curve in scikit-learn
