1
$\begingroup$

I'll try to keep my question as short as I can:

For my study I will include 30-40 patients. For each patient, I will already have retrospective data for creatinine, a substance found in urine and blood.

I will look at the variance in the retrospective creatinine measurements. So for example they might have had 3 urine creatinine measurements in the past: 9,2 ; 6,8 and 8,0. I will apply an intervention to each patient, in the hope to reduce the variance of these values. (for example post-study it would then be 9,2 ; 8,9 and 9,0).

So the setup is pre- and post intervention. The goal is to see whether the intervention decreased the variance in found values pre-post. I also want to measure the variance within group.

After a lot of (confusing) research I am considering taking the standard deviation of the pre and post values, and then employing a paired T-test. I'm not sure if thats a good way to do it, what is your opinion? The substance is normally distributed.

Your thoughts would be very much appreciated. Regards, Daniel

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ If you are using variance as your test statistic then the F-test could suit your need. $\endgroup$
    – user2974951
    Commented Sep 13, 2018 at 12:55
  • $\begingroup$ Why don't you look at doing the Null Hypothesis test? As you're looking to see if an intervention reduces the variance of the values, then this seems like a good contender as your data is already normally distributed. This is also called a P-Valued approach. The following URL should be a good starting point: onlinecourses.science.psu.edu/statprogram/reviews/… $\endgroup$
    – Johnathan Brown
    Commented Sep 13, 2018 at 13:58
  • $\begingroup$ Do I understand correctly that pre-interventions you will have a group of $n$ subjects, and that for the $i$th subject you will have $n_i$ creatinine measurements with individual subject variances $S_i^2.$ Then post-intervention you will have $m_i^\prime$ creatinite measurements on each individual with variances $T_i^2.$ And finally, at the end you want to compare $S_i$ with $T_i$ for each patient? Also possibly, test whether the $S_i^2$s are larger than the $T_i^2$s? // If so, patient by patient comparisons of $S_i^2$ with $T_i^2$ may be problematic unless $n_i$ and $m_i$ exceed 3. ... $\endgroup$
    – BruceET
    Commented Sep 14, 2018 at 0:09
  • $\begingroup$ Finally, when you do an overall paired test on the before vs. after variances, you will not have normal observations. For example $S_i^2$ has a skewed distribution which is related to a chi-squared distribution with $n_i - 1$ degrees of freedom. // It would certainly help if all of the individual patient $n_i$'s and $m_i$s were equal. // I'm not saying this analysis is impossible, but figuring out a reasonable strategy for analysis would be easier if we knew how many measurements on each subject and how many subjects. $\endgroup$
    – BruceET
    Commented Sep 14, 2018 at 0:15
  • $\begingroup$ Hi professor Bruce, your understanding of the study is spot on. I will assume I have 3-5 measurements pre, and 3 measurements post intervention. How would you prove that the variance has decreased post intervention? $\endgroup$
    – Dan Daniel
    Commented Sep 14, 2018 at 14:18

1 Answer 1

Reset to default
0
$\begingroup$

Individual subjects. Suppose you have variances $S^2, T^2$ of two independent samples of sizes $n, m$ from two normal distributions with variances $\sigma^2,\;\tau^2,$ respectively. And suppose you want to test $H_0: \sigma^2 = \tau^2$ against the one-sided alternative $H_a: \sigma^2 > \sigma_2^2.$

Then under the null hypothesis $H_0,$ the statistic $F = S^2/T^2$ has Snedecor's F distribution with numerator degrees of freedom $n - 1$ and denominator degrees of freedom $m - 1.$

Your example. In particular, you give an example in which the pre-treatment creatinine measurements are $(9.2, 6.8, 8.0)$ with $S^2 = 1.44,\; n=3$ and post-treatment measurements are $(9.2, 8.9, 9.0)$ with $T^2 = 0.0233,\; m = 3.$ Then a test for equal population variances (against a one-sided alternative) rejects $H_0$ with P-value 0.016:$

a=c(9.2, 6.8, 8.0); b=c(9.2, 8.9, 9.0)
var(a); var(b)
[1] 1.44
[1] 0.02333333
var.test(a, b, alt="greater")

        F test to compare two variances

data:  a and b
F = 61.714, num df = 2, denom df = 2, p-value = 0.01595
alternative hypothesis: true ratio of variances is greater than 1
95 percent confidence interval:
  3.24812     Inf
sample estimates:
ratio of variances 
          61.71429

If this example of the differences in sample variances is typical of your results, you should have no trouble finding patients with significant reductions in variance after treatment.

Speculation about 40 subjects. Just as an experiment to see what might happen, here is a brief simulation. Suppose we have $N = 40$ subjects, each with three creatinine measurements pre and three measurements post-treatment. Also, suppose that the population standard deviation pre is $\sigma = 1.2$ and the population SD post is $\tau = 0.5.$ This a somewhat less of an effect than in your example.

Generate fake data roughly to specifications:

set.seed(914)
mu = 9;  theta = 1.2;  tau = 0.5;  n = 3;  N= 40
PRE = matrix(rnorm(n*N, mu, theta), nrow=N)   # matrix of creat. meas PRE 
PST = matrix(rnorm(n*N, mu, tau), nrow=N)     # matrix of creat. meas POST

Find PRE and POST variances for all 40 patients. Show results for first six.

vr.pre = apply(PRE, 1, var);  vr.pst = apply(PST, 1, var)
head(cbind(PRE, vr.pre)) 
                                       vr.pre
[1,]  7.158285  8.671603  9.127335 1.06249629
[2,] 10.036264  9.684452  9.529816 0.06736218
[3,]  7.748063  8.757169  8.073015 0.26532575
[4,]  8.984257  7.174361 11.534081 4.79741228
[5,] 11.920725 10.550296  9.038141 2.07899702
[6,]  7.962571 10.309159  7.835060 1.94065025

head(cbind(PST, vr.pst)) 
                                      vr.pst
[1,] 10.155144  9.470168 8.765562 0.48276728
[2,]  9.207062  8.750427 8.558065 0.11111928
[3,]  9.667394 10.173364 9.222864 0.22617705
[4,] 10.175448  8.414801 8.457564 1.00880558
[5,]  9.090010  9.175255 8.935155 0.01481576
[6,]  8.859023  7.874286 8.718930 0.28379321

Histogram of ratios of POST/PRE Variances for 40 subjects. A large majority of patients have ratios below 1.

hist(vr.pst/vr.pre, col="skyblue2", main="Ratios of 40 Patient Variances")
  abline(v=1, col="red", lwd=2, lty="dotted")

enter image description here

Specifically, 36 simulated subjects showed decreased variability. Under the null hypothesis (that increase and decrease are equally likely), the probability of 36 or more increases out of 40 subjects is less than 0.0005. So the number of increases for the fake data would be statistically significant. 

sum(vr.pst/vr.pre < 1)
[1] 36
1 - pbinom(35, 40, .5)
[1] 9.285122e-08

I would not expect actual experimental results to be quite so easily analyzed or for the P-value to be so small. It seems that neither a paired t test nor a Wilcoxon signed rank test would be appropriate. Data are neither normal nor symmetrical. But other methods are available.

In conclusion, if I have understood your proposed experiment correctly and your example was roughly typical of actual results, then this simulation seems a promising indication your study would be successful.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you Bruce, your thorough response has been very clarifying. If I understand correctly, there are two tests I can apply concomitantly: 1. F-test for equal variances pre-post, to prove change in variance 2. Probability calculation under hypothesis that increase / decrease are equally likely, to prove the probability of generated post-pre being >1. I have one question: you mentioned a mean of 9 in your calculations. How was this determined? I hope I interpreted your post correctly. Thank you again, statistics can be confusing in the beginning phases. $\endgroup$
    – Dan Daniel
    Commented Sep 17, 2018 at 14:46
  • $\begingroup$ My bad, I already understood the mu =9! Was merely generation of fake data $\endgroup$
    – Dan Daniel
    Commented Sep 17, 2018 at 14:56
  • $\begingroup$ Used 9 for my simulation because your two examples had means around 9. For the essence of testing change in variance, exact mean is irrelevant. (In real-life expt, I suppose you would be interested to see a major change in mean. And if treatment drives mean to 0, then variance probably goes to 0 also.) $\endgroup$
    – BruceET
    Commented Sep 17, 2018 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.