0
$\begingroup$

I have some background in stats and math but it has been a bit, and I'm not sure if there is some nuance to this question.

We have 15 chest x-rays for pre and post treatment effect.

We have three radiologists who make measurements on each of these 15 chest x-rays pre and post treatment.

I understand that if there were only one person taking measurements, there would be 15 pre and 15 post measurements, and a paired T-test would absolutely be the right test.

But, since we have three radiologists taking measurements of the same 15 chest x-rays pre and post treatment, is a paired T-test still appropriate? And would it just be a T-test on 45 pre and 45 post treatment X-ray measurements? Or is this sort of a repeated measures or multiple paired T-test sort of thing?

Please let me know if I am being unclear. I would appreciate any information or opinions.

Improve this question
$\endgroup$
5
  • $\begingroup$ Are the three radiologists the only ones of interest, or are you trying to make a conclusion that would apply to radiologists more generally? $\endgroup$
    – Glen_b
    Commented Sep 19, 2021 at 23:15
  • $\begingroup$ Thanks for the question. The radiologists are just the observers taking measurements. The conclusion would apply to the treatment effect more generally. $\endgroup$
    – statsquestion
    Commented Sep 19, 2021 at 23:32
  • $\begingroup$ There might be an argument for using some form of random effect term in the model. $\endgroup$
    – Glen_b
    Commented Sep 20, 2021 at 0:22
  • $\begingroup$ Are you interested in exploring variability between radiologists? Would it make sense to average the three measurements into one and use a t-test? You most likely need a more complex design that accounts for the radiologists, but just in case, I thought it might be useful to think about these in terms of "replicates" that could be averaged. $\endgroup$
    – Matias Andina
    Commented Sep 20, 2021 at 1:34
  • $\begingroup$ Hi Matias. We were considering doing a paired t-test on the raw 45 pre paired with the 45 post data. The other consideration was to average them just as you proposed. I am not sure which would be more valid statistically. $\endgroup$
    – statsquestion
    Commented Sep 20, 2021 at 19:41

1 Answer 1

Reset to default
1
$\begingroup$

I think that taking 45 pre\post to the paired t-test, can be in some cases, misleading. Take for example a case where 1 radiologist give very different scores (e.g. much higher) from all the others. In this case the results can be skewed towards this radiologist which can be an outlier.

The same goes for averaging each case between the 3 radiologist and go to paired t-test with 15 averages.

One option, in order to get full information, and then decide what to do with it is: Run a 3 paired t-test, one for each radiologist.

Compute the $\mu, \sigma$ and t-test score for each. If they all agree then it is easy. If not, you can compute if there is a statistical significance difference between the different radiologists (i.e. are their $\mu$ are statistically significant different).

Finally, if there is a contradiction between them, it is up to you to decide what to do. Do you take the majority vote?, do you have other considerations like: do you trust one more than others etc.

Improve this answer
$\endgroup$
4
  • $\begingroup$ +1. I agree with @ofer-a that running 3 paired t-test for each radiologist should be favored in the first place to understand better the situation. $\endgroup$
    – Pitouille
    Commented Sep 21, 2021 at 10:23
  • $\begingroup$ Thank you guys so much! My team and I agree with what you are saying. We intend to run a paired t-test for each radiologist. For inter-reader (radiologist) agreement, we were planning on running a one-way anova on the pretreatment data and an additional one-way anova on the post treatment data. $\endgroup$
    – statsquestion
    Commented Sep 28, 2021 at 22:17
  • $\begingroup$ When running three separate t-tests, do you think it would be necessary to perform a bonferonni correction on the alpha values (.05 / 3)? $\endgroup$
    – statsquestion
    Commented Sep 28, 2021 at 22:19
  • $\begingroup$ @statsquestion regarding bonferonni correction - that's a good point, it depends of how you will eventually use the separate t-test. If you will use it as: $P(one of them pass | H_0) <= \alpha$ then yes, the correction is needed to 0.05/3. $\endgroup$
    – ofer-a
    Commented Sep 29, 2021 at 10:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.