0
$\begingroup$

I have an experiment where I am trying to determine whether the answering behavior of participants can be explained as random answering. I.e. every participants will have to answer multiple questions with two options. The experimental hypothesis suggests a certain pattern, i.e. that participants will not just answer randomly. However I am unsure how to test for this.

The main problem is, that I have multiple answers for each participants, so I am unsure whether I should expect these answers to be independent. If I could take them to be independent, I could easily test how likely it is for the seen data or a more extreme pattern to emerge just by chance. If I have to take into account that the answers for each participants are not independent, I have no clue how to analyse this.

Is there some Analysis in SPSS which allows me to analyse this kind of data?

|cite|improve this question
$\endgroup$
  • $\begingroup$ What test would you use if †he questions are independent? Can you test to see if the answers to the multiple questions are correlated? If they are not correlated, then you might proceed as you describe. BTW, do you have any theoretical or other reason to help you predict the proportion of participants who will chose option A rather than option B (assuming we call your two options A and B)? $\endgroup$ – Joel W. Oct 15 '12 at 22:12
  • $\begingroup$ @Joel: If they are independent I would just check the probability that as many or more than the seen responses are in the appropriate (skewed) direction. I only have a minor clue that some participants may be more susceptible to the effects studied than others, however there is no indication other than the answer as to which ones. I am not so much interested in differences between participants as in differences between answering behavior of the same participants in different settings. $\endgroup$ – LiKao Oct 16 '12 at 8:41
  • $\begingroup$ Are you interested in testing the null hypothesis that the the probabilities of choosing A and B are equal? $\endgroup$ – Joel W. Oct 16 '12 at 12:42
  • $\begingroup$ @JoelW.: Yes, exactly. $\endgroup$ – LiKao Oct 16 '12 at 14:05
  • $\begingroup$ What are the choices A and B? Do they have meaning? Is choice A the same for all questions? If so, to test the null hypothesis that the the probabilities of choosing A and B are equal, you might just get the percent A responses for each person and then do a z test to see if the mean is significantly different from .5. $\endgroup$ – Joel W. Oct 18 '12 at 0:40
1
$\begingroup$

Have you checked item response theory? There are models in psychological/educational testing that allow you to estimate a "guessing" parameter and test its significance.

RE: independence. For these models, you assume that the probability that person i answers question j correctly is a function of the difficulty of question j and the overall skill of person i. Each person's responses are thus assumed to be conditionally independent. If the test itself is formative - so that a person's probability of answering correctly increases with the questions answered - then a different model would be appropriate - and I have no idea what that model would be.

Dimitris Rizopoulos wrote a package for R to analyse test item data called ltm. His excellent article about the models is available here:

http://www.jstatsoft.org/v17/i05/

I don't know how you would do this in SPSS, but Rizopoulos' article is a good introduction to the available models, regardless of software implementation.

For SPSS, I would check the help files for Rasch models or Latent Variable Models and see what they have.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.