3
$\begingroup$

My question is straightforward: I want to do statistical classification in a database, mostly of floats, but one of the fields of the data is a big text. By big I mean up to 1000 words.

What is the best way to deal with that?

UPDATED: It's supposed to be supervised learning: I have some rows that have the values "Closed" and "Not Closed", and want to estimate this row for the other ones.

$\endgroup$
2
  • $\begingroup$ Could you describe what do you mean by 'classification'? Usually classification is understood as a supervised learning task, while in your case you seem to talk about unsupervised learning (clustering)? $\endgroup$ Jun 26, 2012 at 5:57
  • $\begingroup$ @danas.zuokas: updated the question! $\endgroup$
    – Lucas Reis
    Jun 26, 2012 at 12:17

1 Answer 1

1
$\begingroup$

You need to quantify text, that is to create additional variables from that text which would help classification. These might be the length of a text, the frequency of occurrence of certain words or phrases (like in spam filtering) and so on. It might even be the whole document-term matrix, but in this case you will have a lot of non informative variables so you need to think of some kind of regularization.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.