I am new to text mining/classification, and really want to learn more from this community. My data are open-ended survey responses (n=about 6,000) in which the respondent described what happened at the time of an accident. Could you recommend a few ways to extract and classify information from the narratives, using either unsupervised learning or supervised learning methods?
For me, the motivation for this exercise is to find a reliable way to quickly classify the circumstance of the accidents (e.g., by cause, location,..etc.), before implementing a full-blown manual coding plan, which is often laborious.
One of the challenge is that the response is not long, and the 'document-to-term matrix' (DTM) is fairly sparse. On average the respondent used 12 words to describe the circumstances, and the non-sparse entries in the DTM are only 41070 out of 33993840, or 0.1%. In other words, sparsity of the DTM is nearly 100%.
So, when I treated each response as a "document" and apply conventional topic models to the collection of 6,000+ responses, the results were not that great. Although the words within each topic appeared to be semantically coherent, I didn't see much difference in the posterior probabilities (as generated by the "topics") across different words in the text. In other words, the modeling can hardly tell us which "word" is more likely to belong to certain topics than others.
Other people in the forum have recommended a modified version of the conventional topic models which can deal with "short documents". Do you think this is the way to go? Or should I continue the exploration along the line as described here?
