3
$\begingroup$

For example, if I am a scientist evaluating 2 methods to determine blood glucose and I want to compare if one is more variable, I would take, say, 6 samples from each person (subject) and use Method A on 3 and Method B on the other 3. So now I have data that looks like

Subject Replicate Method Glucose
1       1          A      80
1       2          A      80
1       3          A      79
1       1          B      90
1       2          B      91
1       3          B      93
..... etc

and I can calculate the standard deviation within method for each subject and then compare them for Method A vs. Method B. What test is appropriate for this? I think ANOVA is not because it is a parametric test and if I want to know if one standard deviation is greater than the other, we violate homoscedasticity.

Thank you.

$\endgroup$
2
$\begingroup$

The two existing answers suggest the $F$-test of your two variances. This is sometimes referred to as Hartley's test. However, this test will only perform well if the distributions are truly normal. It isn't very robust. It is generally recommended that you test the absolute deviations of the data from their means or medians with a $t$-test (or an ANOVA, if there are >2 groups). It may help to read my answer to: Why Levene test of equality of variances rather than F ratio?

That said, it sounds like your situation may be about characterizing the agreement between the two measures. It may help you to read my answer to: Which test do you suggest to use to catch the differences in paired sample.

$\endgroup$
1
$\begingroup$

You can use an F-test of $H_0: \sigma_1^2=\sigma_2^2$ against $H_a: \sigma_1^2\ne\sigma_2^2$. Sometimes this is written in terms of ratios of variances: $H_0: \sigma_1^2/\sigma_2^2 = 1$ against $H_a: \sigma_1^2/\sigma_2^2 \ne 1.$

However, you will find that three observations in each group (method) is usually not enough for helpful testing---unless the population variances are hugely different.

For example, there is no statistically significant difference between variances in your two samples, according to the procedure var.test in R.

x1 = c(80,80,79);  x2 = c(90,91,93)
var.test(x1, x2)

        F test to compare two variances

data:  x1 and x2
F = 0.14286, num df = 2, denom df = 2, p-value = 0.25
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
 0.003663004 5.571428571
sample estimates:
ratio of variances 
     0.1428571 

If there is a 4:1 ratio of population standard deviations (that's a 16:1 ratio for variances), then the power of this F-test (ability to reject $H_0,$ indicating a significant difference) for only three observations in each group is less than $0.3 =30\%.$ (Such F-tests are notorious for their poor power.)

set.seed(2020)
pv = replicate(10^5, var.test(rnorm(3,0,4), rnorm(3,0,1))$p.val)
mean(pv <= 0.05)
[1] 0.28955

Ten observations in each group would give power above 95% detecting such a large difference between population variances. (There are online 'power and sample size' procedures for this test, and many statistical program also have such procedures.)

set.seed(915)
pv = replicate(10^5, var.test(rnorm(10,0,4), rnorm(10,0,1))$p.val)
mean(pv <= 0.05)
[1] 0.97468
$\endgroup$
0
$\begingroup$

(You're so close when you mention ANOVA!)

Remember that ANOVA works by comparing two variances: the "between" variance and the "within" variance. Then we calculate the F-statistic and compute a p-value.

While we use the F-test to do ANOVA and compare means, the F-test is a test of variances (just in a particular way when we do ANOVA). We can use the F-test to compare any two variances. Then, if we reject that the variances are equal, we reject that the standard deviations are equal.

(Do not make the mistake of thinking that the F-test compares that variances of multiple groups, however.)

I am concerned about you taking multiple measurements from individuals, however, as that violates the independence assumption of most tests.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.