0
$\begingroup$

Can you do IRT, CFA and/or Rasch if your data has items with large ceiling effects (i.e., no one was choosing the last two options of a 5-point likert scale)? Is it appropriate to do?

$\endgroup$
  • $\begingroup$ If no one chose the last two items, does that mean no one chose 4 & 5, or no one chose 1 & 2? $\endgroup$ – Heteroskedastic Jim Oct 31 '18 at 12:12
  • $\begingroup$ In a 12-item measure, no one chooses response options 4 and 5 (which is "severe" and "very severe") on four of the twelve items. This would be at Week 12 only (due to treatment benefit occurring over time for some of the items). At Baseline all response options are chosen. $\endgroup$ – Jane Oct 31 '18 at 18:47
0
$\begingroup$

If you have large ceiling effects, there's nothing you can do about it. With the more sophisticated measurement models, you'll be able to see that the measurement quality deteriorates for those with high scores as you have little information to differentiate them.

If the test is to detect whether an individual knows certain things, ceiling effects are fine; they show those individuals know those things. However if your goal is to measure across the sample under study, then there's no solution.

The nice thing about Rasch/IRT is the process of running and diagnosing the model will reveal these problems to you. EFA can give you similar information, but information about ceiling effects is usually hidden behind many layers; EFA emphasizes other features of the data.

So all you can do is measure with the caveat that for those of high ability, you're struggling to differentiate them properly.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Let's say that there is a treatment benefit that is causing the ceiling effect for items. At baseline, ALL the response options are being chosen for each item, but by Week 12 you have the ceiling effect due to health improvement and the lower extreme response option is no longer being used on SOME items. I know you can do some sort of longitudinal IRT, but can it also be appropriate to just run the IRT/Rasch at each timepoint? Do we need to collapse response categories for none or certain or all items at each timepoint? $\endgroup$ – Jane Oct 30 '18 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.