0
$\begingroup$

I'm currently writing a critique on an RCT and my essay should include a report on the findings of the study. I have very minimal knowledge on interpreting statistics and papers.

I'm confused on how to interpret the confidence intervals for these results. I do know that the two-tailed p-value (set at <0.05) is statistically significant, but I'd appreciate any help and explanation about CIs.

"A significant interaction effect was observed for total fatigue (F = 7.63;P = .001). Post‐hoc ANCOVAs indicated that, after the intervention, the telerehabilitation group improved total fatigue perception significantly (P < .001) compared with the control group. This improvement was maintained after the 6‐month follow‐up period (P = .002) (Fig. 4). The ES was large after the intervention (d = −0.89; 95% CI, −1.30, −0.48) and was moderate after follow‐up (d = −0.74; 95% CI, −1.19, −0.29)."

(mean ± SD (95% CI for the mean)

For full article (link does not work) search: Telehealth System: A Randomized Controlled Trial (Galiano-Castilo et al., 2016)

$\endgroup$
  • 1
    $\begingroup$ If this is homework, please add the self-study tag and tell us what you've thought/done so far. $\endgroup$ – StatsStudent Jan 9 '19 at 1:24
  • $\begingroup$ Also, need to know what the scale is that they assessed peoples fatigue on to make sense of the numbers. $\endgroup$ – André.B Jan 9 '19 at 2:15
  • $\begingroup$ @André.B If you mean the outcome measure, then it's the Piper Fatigue Scale. $\endgroup$ – user233463 Jan 9 '19 at 2:33
0
$\begingroup$

As mentioned in the comments, this seems like self-study. But generally, sometimes people have conceptual difficulty with confidence intervals, in which case I hope the following can help:

A confidence interval (which is nice to see required) is a range of values that with a (95%) chance includes the population coefficient (interaction coefficient here). When we talk about significance, we really are trying to infer from our sample to the population. We can never do that with complete certainty (unless we actually have the entire population to work with). So to find that something is significant is to say that with a certain chance of being wrong (say 5%, 1% etc...), we can reject a NULL hypothesis that normally says that the population value (coefficient, mean, whatever) is equal to zero. So if a coefficient is significant at a p=0.01 level, we assume that the actual population coefficient is likely different than zero (with a 1% chance we are wrong). Note, that does not mean that the population value is the value of our sample coefficient. So what is the population coefficient? We do not know. But we can estimate a range of values among which the population value is likely (with a 95% certainty say) to be. These are the confidence intervals. So, if the confidence intervals contain zero (i.e., one of the values is negative and one positive or zero), we cannot be sure that the population value is different than zero, and we will get non-significant results.

$\endgroup$
  • $\begingroup$ I think I kind of understand what a confidence interval means (looked up a beginner's guide to it) but I'm just confused by the results in this paper as I've been given two statistics (95% CI, −1.30, −0.48). They're negative, I'm not sure if that affects anything. Apologies if I sound ridiculous, but I've never studied stats, only covered some very basics as my course isn't to do with maths/statistics etc $\endgroup$ – user233463 Jan 9 '19 at 2:59
  • $\begingroup$ I have read that if the mean difference of a CI contains 0, it is not statistically significant. If it contains 1, then it is. But, I'm not sure what the stats from my paper are saying. I literally just need to write a sentence on whether the result is significant or not. $\endgroup$ – user233463 Jan 9 '19 at 2:59
0
$\begingroup$

There are a couple of different types of intervals in statistics. A confidence interval is the result of a confidence procedure. What we really have confidence in is the procedure and not the specific interval.

A 95% confidence interval will contain the true parameter no less than 95% of the time on infinite repetition of the exact experiment with different data. It does not tell you that there is a 95% chance that the parameter is inside the interval this one time. There is either a 100% or a 0% chance it is inside the interval because its location is a fact. You do not know the true state, so you do not know the fact.

The goal of a confidence interval is to tell you how to behave. If you would behave as if the interval were correct, then you wouldn't be a fool at least 95% or the time, or 99% or whatever the desired level of the interval is. Sometimes, though, it will be unfortunate, maybe even very unfortunate that you behaved as if the interval were valid.

In this case, if you behaved as if the change were inside the interval, then you can feel at least 95% confident regarding your actions. At least 95% of the intervals from an infinite number of experiments would cover the true parameter.

The interval is the best interval under the assumptions of the problem to estimate the location of the parameter given the specific sample and the sample space. Considering that there is an infinite number of potential intervals, that is a really powerful claim.

TECHNICAL NOTE There are many possible confidence intervals based on differing assumptions behind the specific problem and the procedures chosen. If you made slightly different assumptions regarding the procedure, then the best interval would change to meet those slight changes in assumptions. Those issues are omitted here as they involve math you have not been taught. If you had been taught the math, then you would not ask this question. You would have asked a different question instead.

$\endgroup$
0
$\begingroup$

For starters - an ANCOVA model is inappropriate for those data. The Piper Fatigue scale is a discrete bounded set, meaning that it can only take values between 1 and 10, and only integers (whole numbers). The assumptions underlying ANCOVA is that the errors are normally distributed meaning that they are unbounded (range between infinity and negative infinity). Moreover, it assumes that a value of 2 is twice that of a value of 1, or that 10 is twice that of the feeling at 5. I personally wouldn't put much faith in any of those results. It also looks like they would have needed a repeated measures design if they want to look and pre- and post-treatments - I skimmed the paper but it doesn't mention it if they did. Lastly, the made no corrections for all of their post-hoc tests. They were almost guaranteed to find a significant result by chance alone.

Regarding your question about the CI:

95% CI = mean ± SEM × ~1.96

where SEM = Standard error of the mean. The 95% interval indicates that there was a drop in fatigue score, on average, after treatment by somewhere between 0.48 and 1.3 after intervention and then a similar drop by between 0.29 and 1.19 in the follow-up treatment. There is a lot of ambiguity in that though - this paper shouldn't have been published, as is, in my opinion.

If you have the time, and you want to better understand where those researchers went wrong I would recommend reading this paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy