4
$\begingroup$

I have run a study comparing if a cheap ultrasound device (T) is as efficient at determining osteophytes as an expensive device (V). There was four examiners who reviewed each participant (ID) and noted their own findings of Yes/No presence of osteophytes in four different views. Yes was given a value of 1, no was given a value of 0. This is a within-subject design so each participant was exposed to both devices.

I have discovered than a repeated measures anova is inappropriate for this data due to the nominal dependent variable of yes/no or 1/0. What is an appropriate test for this study to compare if there is a difference in device and examiner?

I also ran a wilcoxon test but this will only review one value (Device) and won't consider examiner. I am also not sure if this test is within-subject and taking into account the participant ID?

 wilcox.test(OSTEOPHYTES ~ DEVICE, data = All.A, mu = 0, paired = T, alt = "two.sided", exact = F)

Any suggestions for a statistical test which will consider a within-subject design looking at multiple variables (Device and Examiner), would be appreciated.

Data Spread for ID (n=26)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
7
$\begingroup$
  1. You have a binary response (osteophytes), for which regression is a good general approach.
  2. You have non-independence due to multiple measurements on the same participant and multiple measurements by the same examiner. The non-independence due to participants can be addressed well using s, also known as hierarchical models. Since you have only 4 examiners, which is a rather low number of levels for a random effect, you may want to include examiner as a fixed effect instead. You will need to consider whether including a device x examiner interaction is a good idea or not.
  3. You are primarily interested in establishing that the cheaper device is not worse i.e. establishing . This can be done with logistic regression, and we have some info at this thread: Noninferiority Using Logistic Regression.

So, use a mixed-effects logistic regression with a random effect for participant. Then check for non-inferiority of the cheaper device.

Improve this answer
$\endgroup$
6
  • 3
    $\begingroup$ In addition to considering a mixed model, a fixed model with a robust sandwich covariance estimator might be considered. $\endgroup$
    – Frank Harrell
    Commented Sep 4, 2023 at 11:43
  • $\begingroup$ Thanks for your response. If I was to not consider the interaction effect of device*examiner and just look soley at them seperatley having only one variable then - What would be an ideal statistical test? Friedman, McNemar Chi Square or Wilcoxon? $\endgroup$
    – Kate Atkinson
    Commented Sep 5, 2023 at 5:46
  • $\begingroup$ @KateAtkinson The approach I suggest above (and the one from Frank Harrell) is not only for when there is an interaction. And if the data structure is as you describe, then it is oversimplifying things to look at them separately with one variable in each model. So I would not recommend any of the tests you mention. $\endgroup$
    – mkt
    Commented Sep 5, 2023 at 5:52
  • $\begingroup$ Thankyou for your input. I have started to do a mixed-effects logistic regression basing from examples, I believe I'm on the right track with this code. glmer(OSTEOPHYTES ~ DEVICE + Examiner + (1 | ID), data = All.A, family = binomial, control = glmerControl(optimizer = "bobyqa"), nAGQ = 10) $\endgroup$
    – Kate Atkinson
    Commented Sep 5, 2023 at 6:56
  • $\begingroup$ @KateAtkinson You're welcome, that code seems like a solid start. Good luck! Also, I just noticed that you have only 4 examiners, so I agree that examiner would be better treated as a fixed effect, as you have done. I have updated my answer to reflect this. $\endgroup$
    – mkt
    Commented Sep 5, 2023 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.