0
$\begingroup$

I'm doing a study on whether 2 groups of physicians: radiologists and oncologists differ in the volume of their contours. Each group has 3 participating physicians and all physicians contour a specific target (haven't thought of what target yet, but let's use the brain stem as an example).

My initial thought is to use a two way ANOVA, but I'm not sure which ANOVA to use. Do I consider each radiologist a repeated measure in the radiology group (and likewise with oncologists in oncology group)? Is that my only independent variable? Which test will tell me if the variations are due to each patient being different or a significant difference between the groups?

The hypothesis is that Oncologists will tend to contour a larger volume than Radiologists.

Below is an example of what the data table might look like. Numbers are made up, but size of study is true. P1 through P21 are the individual patients/subjects. Values represent volume.

enter image description here

$\endgroup$
8
  • $\begingroup$ What are P1 through P21? Some discussion about what a "contour" means will be extremely helpful. $\endgroup$
    – Dave
    Commented Nov 22, 2019 at 15:44
  • $\begingroup$ The Hypothesis to be tested is lacking. Is it "radiologist tend to mark larger volumes then oncologists" or is it "radiologist's volume decision on the same target show lower variance/better reproducibility then oncologist's" or what precisely is the hypothesis or null hypothesis? $\endgroup$
    – Bernhard
    Commented Nov 22, 2019 at 16:03
  • $\begingroup$ P1 through P21 would be the patients. and a contour can be thought of as the volume the doctor deems as the brain stem. $\endgroup$ Commented Nov 22, 2019 at 16:05
  • $\begingroup$ The hypothesis is similar to what you said, that the oncologists tend to mark larger volumes than radiologists because of the nature of their work/training. $\endgroup$ Commented Nov 22, 2019 at 16:06
  • 1
    $\begingroup$ How about a random effects model where the volume is a linear model of the physicians tendency to mark large or small volumes and an individual size of each P and a dummy for radiologist? Would a random effects model be within you statistical scope? $\endgroup$
    – Bernhard
    Commented Nov 22, 2019 at 16:09

1 Answer 1

0
$\begingroup$

The measurements need to be treated as repeated measures with each individual. They are a great way to get a better estimate of the contour-volume deviation for each individual, but they don't improve your estimate of the field as a whole by much. At its simplest, all you are doing is a t-test comparing the average score (or deviation) for each individual between the two groups.

For the study that you describe, your sample size is three per group no matter how many measurements you make on each of those three individuals.

To make broad conclusions about the fields, you need more than three individuals from each group. As a simple example, imagine that men and women differ in their contour size. If we assume that 50% of each field are each gender (for simplicity, though I know this isn't true), there is a reasonable probability that you will get an uneven number from each gender in each group. Then, the effect you would actually be measuring is gender, but you might misinterpret it as being representative of the field as a whole. Repeat this concern over every potential confound (which college they attended, where they attended med school, age, handedness, etc.) and you can see the risk you take in over-interpretting a result.

On top of that, you may simply sample (by chance) from the low or high end of the distribution in each group. The power of the study is also low, which would make it difficult to interpret a negative result.

$\endgroup$
4
  • $\begingroup$ I originally thought that each patient could not be treated as repeated measures since each patient has a different organ size. So wouldn't treating them as a random effect account for the variation that you mentioned (ie.gender)? I also assumed that the ANOVA test would tell us whether the variation is due to the groups or due to the variation within the subjects. $\endgroup$ Commented Nov 22, 2019 at 16:19
  • $\begingroup$ So say I get a larger sample size for each group (ie. 10 observers for each group). What ANOVA test would I be running? and if I can't get to that sample size, is the t-test the best metric to use in its current state? $\endgroup$ Commented Nov 22, 2019 at 16:26
  • $\begingroup$ Each patient is a repeated measure within the individual doing the scoring, and yes, they likely should be treated as random variables (though that just impacts whether or not you are interested in their effects). In effect, what you would be measuring would be how much (on average) each scorer deviates from average. Your linear model would be Score ~ Patient + Group where you treat patient as a random variable and the group represents the groups of interest; or Deviation ~ Group where Deviation already corrects for the patient effect (this is, effectively, the same model). $\endgroup$ Commented Nov 22, 2019 at 20:19
  • $\begingroup$ Yes, the ANOVA would tell you if the groups differ. My point was that the groups may differ for reasons that have nothing to do with your metric of interest (specialty) and that the risk of this is much higher with a smaller sample size (because confounds are more likely to randomly aggregate in one group or the other). $\endgroup$ Commented Nov 22, 2019 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.