I am conducting a psycholinguistics study for a term paper in which I investigate how young children interpret "ambiguous" sentences such as "The dog ate the food on the table" in my native language.

What I mean by ambiguity is that "on the table" could be referring to the location of the food (noun-location), or it could refer to the location where the dog performs the action of eating (action-location).

I am trying to find out whether children have a bias towards one of these interpretations, i.e., whether there is a difference between the frequency of "noun-location" and "action-location" interpretations.

I have tested all children with 4 such sentences in total. So the number of every individual child's total noun-location and action-location interpretations vary between 0 and 4, depending on their biases. (e.g., a child who had a noun-location bias could have interpreted all sentences like that, and then his/her score for noun-location would be 4, while his/her score for action-location would be 0. But if a child gives random responses, s/he may have interpreted 2 sentences as having noun-location and 2 sentences as having action-location, etc.)

I have detected that there is a considerable bias towards noun-location interpretations at the group level, but I'd like to show that this is statistically significantly different from chance as well. (The group mean is somehwere around 3.4 over 4 for noun-location readings, so I anticipate that it should be indeed significantly different from 2 over 4, which we would expect if the interpretations were completely random.)

But which test would be appropriate to test my data against chance here?

  • $\begingroup$ Have you looked at one-sample Wilcoxon test or one-sample sign test? $\endgroup$ – Sal Mangiafico May 15 '18 at 17:27
  • $\begingroup$ I like @michaelmuller's t test idea, but I worry about the power of the test. If you have only 20 students it will be difficult to distinguish between a population bias of .4 or .6 (small/moderate bias) and .5 (no bias). If you have 40 students, then that kind of distinction will be easy to make. (Also, if you had 5 questions instead of 4, that would improve the power for 20 students and make it very good for 40.) [Especially for 20 students, it is a good idea to simulate power values, because it is not obvious the t statistic will really have a t distribution.] $\endgroup$ – BruceET Jun 23 '18 at 23:23

I suggest to begin by testing each of your 4 sentences separately. When you think of the problem that way, then (I think?) each sentence has only two outcomes. You could propose a null hypothesis that children answer by chance. If you find many more children answering one way than the other way, then that suggests that the null hypothesis is incorrect. You can use the binomial test to evaluate your hypotheses (https://en.wikipedia.org/wiki/Binomial_test).

If you start this way, then you will be able to see if all of the sentences have similar outcomes. Then you could add up the results for each child, as you described. If you have four sentences, then the possible outcomes are 0, 1, 2, 3. The null (chance) hypothesis could be that each child should have a score of 1.5 (midway between 0 and 3). You can use a one-sample comparison-against the mean test to evaluate this. The formal way to do this is to test against the expected value of the mean. A simpler way is to subtract 1.5 from each child's score, and then do a one-sample test against the null hypothesis of zero. You could use a one-sample t-test (https://en.wikipedia.org/wiki/Student%27s_t-test).

  • $\begingroup$ If I understand correctly, there will be 4 questions with outcomes 0, 1, 2, 3, 4. $\endgroup$ – BruceET Jun 23 '18 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.