2
$\begingroup$

Suppose I have a bunch of text documents. Let's say articles read by both John and Joann.

I want to kind of classify a new text document into either "preferred by John" or "preferred by Joann". Basically highlighting a bias towards one of them.

I thought of doing typical TF-IDF feature vector from the docs and running a binary classification algorithm to classify a document as either of the two classes.

However if I want to highlight those certain phrases/words in the documents which make the document have a "preference/bias" towards John or Joann, how do you think on can approach this problem.

My initial idea is to build a parametric model (like linear models etc) which can give me a weight vector corresponding to each feature in TF-IDF vector.

Then looking at the weights of these features one can say which weights cause the dependent variable (this would be John or Joann) to be either one of them. It can be a logistic classifier and then one can look at odds ratio to get the phrases/words which the algorithm thinks is causing the bias in documents.

My concern is 3-4 words might not tell if a document is biased towards John or Joann and that causes John to prefer one such article and Joann to prefer other set of articles. Also, what makes someone prefer an article might not solely be dependent upon the words/phrases but also maybe how the story or article has been written, the domain of the article etc.

Do you think this word feature weights is a good rational idea to get the bias preference sense?

$\endgroup$
0
$\begingroup$

Whether looking at 3-gram/4-gram is enough to tell if a document is biased towards John or Joann can be quantified by looking at the classification performance. Such n-grams may not capture very well the quality of writing (though they could still produce a decent approximation {1}) but it should capture the domain well.

FYI:


References:

Early research on the automatic evaluation systems appeared fifty years ago. One of the notable projects was the Project Essay Grade, led by Ellis Batten Page at Duke University (Page, 1968). His work was based on the use of stylistic features of the response of the learner, such as word size and number of prepositions, to predict the human note of correction.

$\endgroup$
  • $\begingroup$ @Frank Thanks for the answer. However my question is, how can one just use a classification algorithm and say the language is biased towards a particular gender in the text?. If we have labels which say that the language is biased then also, it's not the whole language which is biased. It might be only few phrases or words which would be biased. However if you convert the text to TF-IDF or n grams and then classify the labels, you would get normal neutral words also coming out as significant in your model which would have some weight to it. Like words like "new" etc. So I am not sure this work $\endgroup$ – Baktaawar Mar 31 '17 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.