1
$\begingroup$

My survey data includes a battery of 36 Likert-scale questions, which are the IVs of interest in my analysis. Respondents rank how "alike" they are to an individual being described in each item. However, each trait being described is generally desirable, so responses mostly fall on the upper end of the scale. I always knew this was an issue, but I'm starting to think it is more important than I realized. Today I was circling back to an earlier part of the analysis and ran a Spearman correlation matrix that also included all of these variables--all of them have significant positive correlations with each other. I take this to mean that people who gave higher responses tended to always give high responses to every item.

So...what to do? Some quick Googling found some papers on measuring extreme response style (that's what this is, right?), but it wasn't clear to me what one should do about it. Does this require creating a model of the underlying tendency to give extreme responses, or are there simpler solutions like e.g. using each respondent's mean response to adjust their responses?

Is IRT (something I know approximately nothing about at this point) some help here?

Are all the logit models (DV is a hypothetical decision respondents have to make) I ran using these IVs junk now?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

It's hard to do anything about extreme response styles post-hoc because it's difficult to parse out the extreme styles from those who generally were in the upper latent trait realm.

If there were items that people generally should have responded lower to then you could use these as flags to indicate acquiescent response bias if indeed they were associated with such patterns, and simply remove these individuals because it was unlikely they were responding lawfully or meaningfully.

This approach is similar to 'lie detecting' items in some standardized psychometric instruments which help to identify whether responders were answering consistently or trying to match some underlying ideal that make them look more favourable (like being 'too likeable').

Improve this answer
$\endgroup$
3
  • $\begingroup$ I actually found an answer that at least has the virtue of being what others have done before in my specific area. First I drop anyone who uses any of the scale answers too often (60% or more for any response, 40% or more for the top of the scale), then I ipsatize responses by subtracting the row mean for each individual from their responses. This introduces some serious complications, but dealing with them is better than scrapping my dissertation. $\endgroup$
    – Owen
    Jan 29, 2016 at 20:30
  • $\begingroup$ I'm unclear why ipsatization is required (you must have some question about relative deviation patterns rather than between group differences), but the removal of extreme cases with that method may be justified on rational grounds; especially if others have used this approach before. $\endgroup$
    – philchalmers
    Jan 29, 2016 at 20:45
  • $\begingroup$ Yes, this is usually applied to groups with a unique response style in cross-cultural research. I think the logic is to extend that to individual response styles. $\endgroup$
    – Owen
    Jan 29, 2016 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.