Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'd like to be able to design a classifier that can distinguish between different types of conversations (not necessarily tell anything about mood, sincerity, or outcome, that's a bit too far fetched).

To know, for example, that among 50 samples of conversations, 10 involve both parties seeking information about a future event, 30 seem to have no goal, and 10 involve one party seeking information from another about a past event (really the algorithm would classify these as types I, II, or III without regard for the actual circumstances).

In other words, the order of the speakers would matter along with the content, perhaps helped along by seeding the algorithm with certain keywords.

Is there a system of classification that could perform this task with a fairly high degree of precision?

share|improve this question
    
to clarify, is this text data, or audio data? –  tdc Feb 17 '12 at 14:38
    
@tdc Text data, sorry –  jonsca Feb 17 '12 at 18:48
add comment

1 Answer 1

up vote 4 down vote accepted

This is how I would approach it. You actually need to check if a text is in class I or III (else it would be class II).

  • First, define a bag of words for classes I and III. You can manually do this
  • For each text, calculate the tf-idf for the words in these two classes and sum it (get two sums).
  • If some of these two sums is above some predefined threshold then it belongs in that class.

If you have a learning dataset big enough, you can easily find out what are the two bags of words, as well as the two thresholds for them.

share|improve this answer
    
I was just going to check out tf-idf from reading your question. Sounds promising. –  jonsca Feb 19 '12 at 2:19
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.