Hi all I am a new to machine learning. I am trying to use ML to predict the classifications of social media comments from users (text).

There are 2 sets of data, data set one includes 5,000 comments (with 30,000 features) and

data set two includes 20,000 comments (with 90,000 features).

I am building Naive Bayes Classifier for two sets of data respectively , and there are 4 classifications.

May I ask about the minimum sample size on the two datasets? Thank you very much!

  • What criteria are you using to determine the sample size? Without specifying criteria there is no way to answer the question. – Michael Chernick Apr 6 at 3:35
  • thank you Michael! Could I ask about what criteria should I consider? Is that meaning like "predictive power and "features ratio"? – Anna Apr 6 at 3:47
  • I am not sure what it is for this application. To give you an idea if you are estimating a model parameter you might be generating a confidence interval the criterion would be the width of the confidence interval for a given confidence level. – Michael Chernick Apr 6 at 3:53
  • 1
    Many thanks! Since I am really new in ML, I have found a formula from "Mail and Internet Surveys" (Dillman, 2000) which indicates that 880 sample from data set 1 (with 5,000 comments) and 1014 sample from data set 2 (with 20,000 comments), where the confidence level is 95%. But I do not know the theory can be applied to ML or not. Could I ask that is this credible in ML sampling? Moreover, I have set the target of the classifier's predictive power 75%. By using the previous no. of sample I can achieve it. Sorry for such questions because this is the first time for me being in touch with ML. – Anna Apr 6 at 4:18

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.