0
$\begingroup$

Can I say that Language Identification is part of Text Mining? If yes, what is exact relationship between these two terms?

$\endgroup$
0
$\begingroup$

Yes you can! Language identification (LI) is a pure multi-class classification problem, while text mining (TM) can be understood as a family of concepts (including text classification, clustering and other machine learning but also natural language processing concepts).

BTW: LI can be seen as a (near)solved task, as existing approches achieve very high accuracies (to my best knowledge around 95-100% even for exotic languages). However, this does not hold in general for TM, as it highly depends on the involved tasks.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thansk for the answer! So Language Identification is a text classification application and text classification is a component of text mining. You say that LI can be seen as a near solved task, but if we are dealing with small texts, has this accuracy (95 ~~ 100%) the same efficienty? $\endgroup$ – Dayvid Welles Sep 9 '15 at 15:55
  • $\begingroup$ Yes! Even if the document length is short (500 characters), LI methods are still able to classify documents correctly. I'm writing this out of experience, where in a (non-published) paper i have applied classification regarding 50 languages and was able to achieve for 300 texts written in 50 languages (6 texts per language, 500 characters per text) 94.67 % accuracy. $\endgroup$ – Unhandled exception Sep 9 '15 at 16:05
  • 1
    $\begingroup$ Wow! Very nice! Congratulations! I'm going to start my course completion assignment and i wanna write something about Language Identification, focusing in portuguese, but now i think that i'm going to change for something else, 'cos LI has a high accuracy. Well, thanks for everything! $\endgroup$ – Dayvid Welles Sep 10 '15 at 0:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.