4
$\begingroup$

Disclaimer: I'm no expert in the field, this question could be axiomatically flawed.

When programming a very basic regression based machine learning algorithm one uses a number of variables; it seems to me that when applying this to natural language processing the number of variables would be so large that the training set would be unreasonable. I've heard that a good rule of thumb for training set size is 5 - 15 times the number of variables - even with a relatively simple application that's many thousands of words - many millions of 2 word 'n-grams'.

Overall question: How would one apply machine learning when one of the inputs is a body of text?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

There are other techniques, but the simplest one is to start with dimensionality reduction - e.g. Principal Component Analysis. After reducing the dimensionality you can employ usual ML tools for doing classification, regression, etc.

Here are a few articles that may give you more information about the process

  • Martins, C. A., Monard, M. C., & Matsubara, E. T. (2003). Reducing the dimensionality of bag-of-words text representation used by learning algorithms. pdf
  • Shafiei, M. et al (2007). Document representation and dimension reduction for text clustering. pdf
  • Penagarikano, M., Varona, A., Rodríguez, L. J., & Bordel, G. (2011). Dimensionality Reduction for Using High-Order n-Grams in SVM-Based Phonotactic Language Recognition. pdf
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the great answer. Any time I get an answer with references it's always more helpful! $\endgroup$
    – FraserOfSmeg
    Commented Jan 11, 2015 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.