# Estimating the mean and variance for a set of probabilities (bounded by 0 and 1) based on Image Segmentation results

## Data

My data is from a set of images wherein I am computing the Sensitivity of an image segmentation algorithm. Sensitivity is computed as:

$$Sensitivity=\frac{TP_{pixels}}{TP_{pixels}+FN_{pixels}}$$

Here is a zoomed in section of an example image: In this data there is a nested structure where there are multiple images of a duck nested within various ducks such as follows: A histogram of the Sensitivities (SE) computed from each image looks as follows: And again, but each type of duck is color coded: ## Overall Goal

I would like to compute the overall sensitivity and confidence intervals of the Segmentation Output across the entire data-set while taking into account the correlations within ducks.

## What I have tried

### First Approach (simple average)

Simply averaging all the Sensitives from each image results in a mean of: 0.810 (0.021 standard error).

### Second Approach (linear mixed-model)

Using the lme4 package I tried a linear mixed-effects model with the formula:

library(lme4)
se.model1 <- lmer(Sensitivity~(1|duck_id), data=sub_d)


The output is as follows:

Random effects:
Groups   Name        Variance Std.Dev.
duck_id  (Intercept) 0.09669  0.3109
Residual             0.04350  0.2086
Number of obs: 209, groups:  duck_id, 19

Fixed effects:
Estimate Std. Error t value
(Intercept)  0.70478    0.07351   9.588


This results in a mean Sensitivity of 0.705 (+/- 0.074 standard error).

### Third Approach (logistic mixed-model)

se.model2 <- glmer(Sensitivity~(1|duck_id), data=sub_d, family='binomial')

Random effects:
Groups  Name        Variance Std.Dev.
duck_id (Intercept) 2.899    1.703
Number of obs: 209, groups:  duck_id, 19

Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept)    1.814      0.476   3.812 0.000138 ***


This results in a mean Sensitivity of 0.860 (0.792 - 0.908 standard errors).

### Question

Each method has a different result and I am confused as to which methods is the most appropriate for this situation.

Do any of these approaches find the correct estimate of the mean and variance?

• What kind of variable is SE ? It's very odd that you've fitted a linear model and a logistic model with the same response variable. Since you have repeated measures then you need to account for that since the observations are not independent. The simple average doesn't do this. Aug 13, 2021 at 22:16
• @RobertLong sorry for the confusion. SE was the vector of sensitivity values computed from each image. I updated the question to change this abbreviation to read Sensitivity instead. The question I have is which model is correct for the outcome variable, the linear or the logistic? Aug 13, 2021 at 22:35
• That depends on what type of variable SE is. If it's a binary variable then fit a logistic model. If it's continuous then start with a linear model. Aug 13, 2021 at 23:08