4
$\begingroup$

one of the reviewer of a paper of mine suggested to perform a homoscedasticity test between the results of two experiments, testing the same thing in two conditions. The experiments consisted in ratings along a 7 points Likert-scale. One of the experiment results were distributed on a large range of values, while the other showed a tendency towards the center of the Likert-scale range (i.e. 3.5). The reviewer argued that the latter behaviour could be due to the fact that participants answered more randomly, and he suggested that to verify this possibility, an homoscedasticity test should be performed comparing the two conditions.

Now, I would like to understand this comment. In relation to my case, what does it mean that the variances of the two conditions are significantly different? What instead if they are equal?

Secondly and more importantly, which test for homoscedasticity do you suggest I perform? I use R. Can you suggest the function more suitable for my case? I saw that there are many.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

I think this question has been addressed a few times before although not in the context of your paper. Even though the likert scale is ordinal, some people here argue that average scores can be looke at as numerical (interval). Ascribing to that theory the variance of the averages has some meaning. Also a Likert scale even though it is discrete the individual scores could be thought of as interval if you believe that respondents think the change from 2 to 3 is the same as from 3 to 4 etc. This may be a stretch. Of course if the raw data were normally distributed you could use the F test for comparing two normally distributed variables. Unless the sample size is large it is difficult to reject homoscedasticity. it has also been pointed out that the F test is not robust to depatures from normality. So it is safer to use a robust test. Levene's test is one of the most common ones and there are modifications to it that have been proposed. Still you may be uncomfortable treating ordinal data as though it was were interval. So maybe a nonparametric test based only on ranking the data or on permutations would be suitable. then you really are asking if the distributions have different scales.

There are apparently three well-known nonparametric tests for equality of scale in two distributions with well-known properties and some new one appearing in the literature. A Google search using "nonparametric tests of scale" turns up a lot of interesting links. There is even one that discuss whether or not Likert scales should be viewed parametrically or nonparametrically with a paper like yours instigating the discussion. Do the search.

But to help out here are links to a few papers. Full pdfs are not available

  1. Mathur
  2. Klotz
  3. Penfield.
Improve this answer
$\endgroup$
12
  • $\begingroup$ Dear @Michael Chernick thanks for your answer. Although it enligthened me, still I am searching the answer to the question I posed: "In relation to my case, what does it mean that the variances of the two conditions are significantly different?". Can you explain me this please? In addition, is the Bartlett test a good test for homoscedaisticity for my case? In R I found that test for homoscedaisticity. Do you have other suggestions? $\endgroup$
    – L_T
    Commented Sep 15, 2012 at 9:12
  • $\begingroup$ More information about your research is needed to answer. Generally, if the difference is significant and if you have measured some attitude, then narrow-ranged case can say that under this condition people are uncertain about their attitude. Another example: people under tranquilizators will choose central options in questions that are measuring some emotional attitude. $\endgroup$
    – O_Devinyak
    Commented Sep 15, 2012 at 10:29
  • $\begingroup$ @Luca Because the parametric ANOVA test requires that the groups each have normal distributions with the same variance checking the assumption can sometimes be important. Basically the F test for homoscedasticity looks at the ratio of sample variances to test whether or not the ratio of population variances differ significantly from 1. The test depends heavily on the normality assumption. So when normality fails robust tests of the equality of variance have been devised. $\endgroup$
    – Michael R. Chernick
    Commented Sep 15, 2012 at 11:26
  • $\begingroup$ More generally, for discrete data and particularly discrete data such as Likert scales which are ordinal nonparametric tests for scale differences have been proposed. For a some probability distributions a variance may not exist but the concept of difference in variation between two such distributions remains valid. So nonparametric tests which look for differences in scale are applied in such situations. Your situation seems to fit this. Robust tests that are still comparing variances will nt be appropriate when the variances do not exist. $\endgroup$
    – Michael R. Chernick
    Commented Sep 15, 2012 at 11:33
  • $\begingroup$ So I think the nonparametric scale tests that I referenced would be most appropriate for your situation. The Barlett test is really similar to the F test because it is also assuming normality and is sensitive to the assumption of nromality in the distributions for the two groups. so it is not as good as robust tests such as the Levene test or the Brown-Forsythe test. I suggest for your to not use the robust test either. Choose from the nonparametric tests for scale differences. $\endgroup$
    – Michael R. Chernick
    Commented Sep 15, 2012 at 11:40
0
$\begingroup$

According to the comments above reported I found that the solution is to use mood.test().

The result is

> mood.test(scrd_non_interactive$Response,scrd_interactive$Response)

Mood two-sample test of scale

data:  scrd_non_interactive$Response and scrd_interactive$Response 
Z = 0.3895, p-value = 0.6969
alternative hypothesis: two.sided

From this I conclude that the variance of the two groups is not statistically different.

Improve this answer
$\endgroup$
7
  • $\begingroup$ That would be fine if the reviewer will buy the idea that a Likert scale is like an interval measure. If he doesn't then you should do one of the nonparametric scale change tests. $\endgroup$
    – Michael R. Chernick
    Commented Sep 15, 2012 at 15:59
  • $\begingroup$ but the mood test is non parametric... $\endgroup$
    – L_T
    Commented Sep 15, 2012 at 16:38
  • $\begingroup$ It tests variances and not general scale when variance is undefined or does not exist. $\endgroup$
    – Michael R. Chernick
    Commented Sep 15, 2012 at 19:21
  • $\begingroup$ Oh no. Then is that wrong? Do you have a name for the test you suggest? I am not able to find in R the right function.... Anyone who has a different suggestion from the Mood test kindly suggested by Fosgen? $\endgroup$
    – L_T
    Commented Sep 15, 2012 at 22:03
  • $\begingroup$ I apologize Penfield mentions Siegel-Tukey, Mood and Normal Scores as scale tests. So you are okay with Mood's test for scale. $\endgroup$
    – Michael R. Chernick
    Commented Sep 15, 2012 at 22:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.