1
$\begingroup$

My research concerns the language of Alzheimer's patients. As the disease progresses, their language becomes more concrete and less abstract - they seem to 'lose' their abstract vocabulary more quickly. Tracking that change over the course of the disease might have clinical benefits.

I have identified a number of factors that measure (to an accuracy of about 85%) the relative concreteness of nouns, within an SPSS binary logistic regression (BLR) model, comprising a constant and four independent variables. The BLR model produces a 'score' for each individual noun: low or negative for abstract nouns, higher and positive for concrete nouns. The objective is not simply to classify the nouns as abstract or concrete, but rather to rank them along a gradient.

To obtain a 'concreteness rating' for a text, I simply calculate the mean of the scores of all the nouns in the text. Although this has given good results in testing, it has been suggested that this is not a legitimate application of BLR (my knowledge of which has been gleaned from YouTube videos).

So - is there a fundamental flaw in my method? And if so, what might be an alternative?

Any help and advice would be very gratefully received.

Kevin

$\endgroup$
1
$\begingroup$

It doesn't seem fundamentally flawed to me. For this to work, you need

  1. A training set of nouns that are coded "abstract" or "concrete".
  2. A BLR model that relates concreteness to your independent variables.
  3. Test this model on a test set of nouns whose concreteness you know.

What the BLR does is estimate the logit (log odds) of concreteness for each noun. So although each noun is either concrete or abstract, the score can be any real number. This gives you your gradient. Alternatively, you could use the associated probability, which has to lie on [0,1], but I think the log odds would perform better.

As far as it goes, your approach seems to make sense. The real test is whether the scores that you get for pieces of text reflect the truth of the matter ... do "concrete" texts seem concrete when read through by a discerning English speaker?

Statistically speaking, this isn't really about binary logistic regression. You are using logistic regression to build a score. Whether this makes sense or not depends on whether the score you get is sufficiently subtle to do what you want.

For example, taking the average of the word scores balances concrete against abstract nouns. Think "Zen and the art of motorcycle riding" --- how would you rate a title like that? You might want to rate abstract nouns higher than concrete ones, for example .. since everyone can use concrete nouns.

Anyway, I would play around with a bunch of weighting options and test them against normal people and Alzheimer people to see what works.

Note: I am assuming here that you are using logistic regression because it is easier to calculate the independent variables than the concreteness or otherwise of English nouns. However, if you have a way of knowing the concreteness, based on some dictionary someone has compiled, or by hand coding, then you are better off basing your score on the binary variable concrete/abstract than on your estimate. The regression model is trying to predict concreteness. It can never do better than actually knowing concreteness. An English prof might actually know where such a word list would be. That's not really a question for this site, however.

$\endgroup$
  • $\begingroup$ Thanks for a very useful reply. The model was trained and tested on substantial datasets (1000 and 2900 respectively) that had been independently rated. $\endgroup$ – user3276135 Sep 1 '15 at 11:01
  • $\begingroup$ If you like the answer, perhaps you could click "accept". $\endgroup$ – Placidia Sep 1 '15 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.