6
$\begingroup$

In “A Comparative Study of Nursing Home Residents …”, Aigner et al. write (p. 20)

Acute visits were significantly higher for the nurse practiticoner/physician team at 3.0 visits per year (± 2.4) versus 1.2 visits per year (± 1.5) for the physician-only group (P <0.0001)

Furthermore, (p. 19)

The statistical analysis was performed using the chi-squared or Fisher exact test for comparisons of percent and Student t test for comparison of means

I always thought that if confidence intervals overlap by that much, differences can’t be significant. Am I wrong?

Aigner, M. J., Drew, S., & Phipps, J. (2004). A comparative study of nursing home resident outcomes between care provided by nurse practitioners/physicians versus physicians only. Journal of the American Medical Directors Association, 5(1), 16-23.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Reporting possible negative number of visits per years in the descriptive statistics looks a bit odd to me "1.2 visits per year (± 1.5)". $\endgroup$
    – user10525
    Commented Apr 20, 2012 at 10:28
  • 1
    $\begingroup$ They're probably not confidence intervals for the mean because, as you pointed out, overlap of $(1-\alpha) \cdot 100\%$ confidence intervals is equivalent to rejecting the corresponding two-sample test of means at the $\alpha$ level (assuming the same distribution is used to generate both, e.g. $t$-distribution, which was probably used here). $\endgroup$
    – Macro
    Commented Apr 20, 2012 at 12:16
  • $\begingroup$ The thought is correct, Macro, but the application seems a little bit off: [lack of] overlap of confidence intervals is definitely not equivalent to rejecting the hypothesis of equal means at the $\alpha$ level. It's equivalent to a much stronger test than that. $\endgroup$
    – whuber ♦
    Commented Apr 20, 2012 at 17:30

2 Answers 2

Reset to default
5
$\begingroup$

They're probably not confidence intervals for the mean, but rather standard deviations from the data, just reported weirdly.

This interpretation is supported by the very confused presentation of results in the second quote. Fisher's exact test is a) not the same as a chi-squared test, b) almost certainly inappropriate given most sampling schemes (both margins are seldom fixed). And if they used both we'd want to know why. Neither test compares percentages, although probably results are ultimately reported in percentages. Finally, applying Students t with such low counts seems risky at best.

From the other side of the paywall its hard to say more, but I think you're seeing a weak analysis ambiguously presented.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Hi, thanks for your answer. It also seems to me that they are reporting something different than SD of the means. However, in defense of Aigner et al., they do not apply Fischers’exact here but Student’s t test. $\endgroup$
    – mzuba
    Commented Apr 23, 2012 at 10:07
5
$\begingroup$

The data aren't normal. I presume that the number of acute visits is an integer, i.e. you can't visit a patient 1.5 times. You either visit them once or twice.

As an example, here are some data:

Mean: 3.2 sd: 2.142
8 8 4 1 2 2 0 2 5 2 3 3 3 1 5 4 4 1 4 2

and 

Mean:1.25 sd: 1.164
4 0 4 1 1 2 0 1 2 2 1 2 1 0 0 0 1 1 1 1

If you performed a t-test, you would get p=0.001

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ At first I thought this reply did not address the question, but then I realized it's pointing out that a t-test is probably inappropriate because the evidence is that the data are (a) few in number and (b) highly skewed. (Normality isn't really an issue with t-tests; normality of the sampling distribution of the mean is what matters.) Interestingly, a permutation test of the mean for these data gives a one-sided p-value of 0.00038 (n=100,000), suggesting a two-sided p-value around 0.0008, which is exactly what a Wilcoxon test produces. The t-test lacks a little power, but seems ok here. $\endgroup$
    – whuber ♦
    Commented Apr 20, 2012 at 17:40
  • $\begingroup$ @whuber How do you perform the permutation test of the mean? $\endgroup$
    – Antoni Parellada
    Commented Nov 23, 2015 at 5:57
  • $\begingroup$ @Antoni Resample repeatedly from the combined dataset. In R this could be done efficiently as x <- scan(text="8 8 4 1 2 2 0 2 5 2 3 3 3 1 5 4 4 1 4 2"); y <- scan(text="4 0 4 1 1 2 0 1 2 2 1 2 1 0 0 0 1 1 1 1"); xy <- c(x, y); x.n <- length(x); s <- replicate(1e5, sum(sample(xy, x.n))); s <- s / x.n - (sum(xy) - s) / length(y) stat <- mean(x) - mean(y); mean(s > stat + 1/2*(s == stat)). It takes less than one second. $\endgroup$
    – whuber ♦
    Commented Nov 23, 2015 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.