I'm working at a project that would like to do topic extraction / classification from data, consisting of various NL sources (tweets, social network updates, pastebin). Data are very diverse in content and lenght, and extremely noisy.

The aim is to predict:

a) pertinence to a given set of categories b) distinguish between relevant / non relevant documents for a given application

I'm not really an expert in ML, coming from a more traditional statistics backgroud. Are there techniques that one could use to get some results in term of prediction, lacking a labeled dataset -- which would be very costly to implement right now?

  • 1
    $\begingroup$ What are you predicting? Topics? Writers/Authors? $\endgroup$ – user145807 May 30 '17 at 20:47
  • $\begingroup$ You can choose a small sample (e.g. 100), label it and then make a forecast for all data. The forecast will not be precise but it is still workable. $\endgroup$ – keiv.fly May 30 '17 at 21:16
  • $\begingroup$ I don't want to discourage you, but in my experience, if the data sources are very heterogeneous, you probably can't easily build one classifier to "rule them all". But whatever, YMMV, as always. :-) $\endgroup$ – fnl May 31 '17 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.