0
$\begingroup$

I'm not big on statistics, so please excuse my ignorance.

I have a video recording that I want to evaluate, I have an algorithm that can transform this video into a time series where I have 0 everywhere except for a couple of frames where an event (A) occurs.

Then I have manual annotations of another event (B) that I think is related to the 1st event (A) (i.e. event B appears shortly after event A).

I want to construct a confusion matrix like this:

  1. Event A is condition, event B is "test"
  2. True positive value is when there is an event A and within 50 frames there is also event B
  3. False positive is when A is not present, but B is
  4. False negative is when A is present, but B is not

Now my problem is about True Negatives. If you do vaccine testing, you have a total number of tests, and you can quantify True Negatives without issue. But what about my case? I either have event A or B but True Negative is by definition everything else in the time series?

Does it make sense to even use confusion matrix?

$\endgroup$
1
  • $\begingroup$ Reading it more carefully, if you're using A as an input signal to predict event B, then your error types seem to be swapped - false positive would be when you predicted B (had seen A) but B didn't happen; and false negative would be when you didn't predict B (didn't see A), but B was actually present. So #3 and #4 would be opposite to what you have written now. $\endgroup$
    – Peteris
    May 20 '14 at 7:11
0
$\begingroup$

Yes, the default value of 'no event found' would be a True Negative, just like any other "needle in a haystack" classification problem.

For binary classification usually you wouldn't call it a confusion matrix, the Precision and Recall terminology would be more clear.

$\endgroup$
6
  • $\begingroup$ Velcome to the site! $\endgroup$ May 19 '14 at 9:55
  • $\begingroup$ Thanks, but the problem I have is that I can not quantify this 'no event found' $\endgroup$
    – mirosval
    May 20 '14 at 5:40
  • $\begingroup$ @mirosval The idea is that you count these events for each point at the original time series, so your example class #4 would be at the points opposite to #2, i.e. when there is an event A, but within 50 frames there is not also event B. $\endgroup$
    – Peteris
    May 20 '14 at 6:42
  • $\begingroup$ @Peteris Ah I should have mentioned that these are rare events (I think that's the term?) i.e I have less than 10 in about 6k frames. $\endgroup$
    – mirosval
    May 20 '14 at 7:04
  • $\begingroup$ @mirosval in many problems such an imbalance is common (or stronger, like 10 in a million). That's the whole reason for treating precision and recall separately, to distinguish between types of error. If you have 10 real events to be detected, then in any case true positives+false negatives should be equal to 10; and you might be interested in what proportion of those 10 events you get correctly. $\endgroup$
    – Peteris
    May 20 '14 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.