# Take into account uncertainty in ground-truth in a classification

I need to evaluate the accuracy of a prediction model (binary classification), given that I have some kind of uncertainty on the ground truth measurement.

The model predicts the occurrence of an event at time T, T+5mn, T+10mn, etc. Unfortunately, the measurements are less frequent, and not aligned. For instance, it can say that an event occurred between T+2 and T+17 (15mn period, not aligned).

If both prediction and measures are negative on a period, then I have true negatives.

But some configurations have uncertainties: if the prediction is negative at T+5,T+10,T+15, but the measurement in the range [T+2,T+17] is positive, I can deduce that at least one of the 3 predictions is a false negative, but maybe not all.

Is there a way to model this uncertainty in the evaluation of the prediction ?

A basic way would be to count the best-case and worst-case false negatives and get a range in the end (and the same for positives). Any better way ?

• In your example, what if the event took place at T+16 or T+17? Then wouldn't you have true negatives? Commented Sep 30, 2022 at 10:16
• If the event occurs at T+16 or 17, it would be measured by the sensor (positive), and yes I would have uncertainty for all 3 predictions. Commented Sep 30, 2022 at 15:45
• Do I understand correctly, that your prediction times and measurement times do not coincide, but you believe that values at nearby times are likely to be equal? And now you think about aggregating the measurements over intervals (periods)? What do you mean by "the measurement in the range [T+2,T+17] is positive"? Does it mean that the average of measurements in this interval is positive? And how do you determine those interval borders? Commented Oct 2, 2022 at 7:59
• The phenomenon has a duration in time. The measurement is done by a in-field sensor, doing an integral over time. The time boundaries of the sensors depends on external factors. The models predicts an intensity of the phenomenon at periodic points in time. Commented Oct 3, 2022 at 11:44