Can I say that Language Identification is part of Text Mining? If yes, what is exact relationship between these two terms?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Yes you can! Language identification (LI) is a pure multi-class classification problem, while text mining (TM) can be understood as a family of concepts (including text classification, clustering and other machine learning but also natural language processing concepts).
BTW: LI can be seen as a (near)solved task, as existing approches achieve very high accuracies (to my best knowledge around 95-100% even for exotic languages). However, this does not hold in general for TM, as it highly depends on the involved tasks.
-
$\begingroup$ Thansk for the answer! So Language Identification is a text classification application and text classification is a component of text mining. You say that LI can be seen as a near solved task, but if we are dealing with small texts, has this accuracy (95 ~~ 100%) the same efficienty? $\endgroup$ Sep 9, 2015 at 15:55
-
$\begingroup$ Yes! Even if the document length is short (500 characters), LI methods are still able to classify documents correctly. I'm writing this out of experience, where in a (non-published) paper i have applied classification regarding 50 languages and was able to achieve for 300 texts written in 50 languages (6 texts per language, 500 characters per text) 94.67 % accuracy. $\endgroup$ Sep 9, 2015 at 16:05
-
1$\begingroup$ Wow! Very nice! Congratulations! I'm going to start my course completion assignment and i wanna write something about Language Identification, focusing in portuguese, but now i think that i'm going to change for something else, 'cos LI has a high accuracy. Well, thanks for everything! $\endgroup$ Sep 10, 2015 at 0:12