My data is from a set of images wherein I am computing the Sensitivity of an image segmentation algorithm. Sensitivity is computed as:
Here is a zoomed in section of an example image:
In this data there is a nested structure where there are multiple images of a duck nested within various ducks such as follows:
A histogram of the Sensitivities (SE) computed from each image looks as follows:
And again, but each type of duck is color coded:
I would like to compute the overall sensitivity and confidence intervals of the Segmentation Output across the entire data-set while taking into account the correlations within ducks.
What I have tried
First Approach (simple average)
Simply averaging all the Sensitives from each image results in a mean of: 0.810 (0.021 standard error).
Second Approach (linear mixed-model)
Using the lme4 package I tried a linear mixed-effects model with the formula:
library(lme4) se.model1 <- lmer(Sensitivity~(1|duck_id), data=sub_d)
The output is as follows:
Random effects: Groups Name Variance Std.Dev. duck_id (Intercept) 0.09669 0.3109 Residual 0.04350 0.2086 Number of obs: 209, groups: duck_id, 19 Fixed effects: Estimate Std. Error t value (Intercept) 0.70478 0.07351 9.588
This results in a mean Sensitivity of 0.705 (+/- 0.074 standard error).
Third Approach (logistic mixed-model)
se.model2 <- glmer(Sensitivity~(1|duck_id), data=sub_d, family='binomial')
Random effects: Groups Name Variance Std.Dev. duck_id (Intercept) 2.899 1.703 Number of obs: 209, groups: duck_id, 19 Fixed effects: Estimate Std. Error z value Pr(>|z|) (Intercept) 1.814 0.476 3.812 0.000138 ***
This results in a mean Sensitivity of 0.860 (0.792 - 0.908 standard errors).
Each method has a different result and I am confused as to which methods is the most appropriate for this situation.
Do any of these approaches find the correct estimate of the mean and variance?