The update of probability distribution given evidence

Question

I've got a problem, which I feel is related to this post. However I cannot really grasp the similarity and adopt it.

I am trying to deploy a model for object detection from video. The model outputs (from each frame) a probability of an object belonging to $n$ classes. For the final prediction I use the class with maximum probability. Usually the system works fine, however at some frames it outputs a different class than "expected".
For example: I have a frame at time $t_{-1}$ showing a back of a car (like a dash cam) - the output is a car with 0.9 probability and a cow with 0.08 probability (the rest of the classes are 0.02 in total). The output of the next frame at $t_0$ (which visually is no different from the one at $t_{-1}$) predicts a car with 0.4 probability, and a cow with 0.5 probability. The later frame at $t_{+1}$ is back at "normal" - similar to the frame at $t_{-1}$.

What I want to do is to use the probability of "neighboring" frames ($t_{-1}$, $t_{+1}$) to change the probability distribution of the middle frame.

For now I thought of three approaches:
1. I will average all three distributions and normalize it afterward.
2. I will multiply all three distributions classwise and normalize it afterwards.
3. I will use a Bayes rule to update the middle frame distribution according to the evidence obtained from neighboring frames.

Problems:
1. This solution basically says that I can tread the middle frame as "corrupted" and discard it, preserving only an average of the neighboring frames. (Well, I don't really want to do that.)
2. This solution seems better, because the neighboring frames "model" the prior distribution. However it still doesn't feel quite right.
3. This solution is quite easy to understand, however I have no idea how to formulate it in a mathematical form. What I get from the Bayes rule is:
$$ P{(t_0 | t_{-1} , t_{+1})} = \frac{P{(t_{-1} , t_{+1} | t_0)}P(t_0)}{P(t_{-1}, t_{+1})} $$ where $P{(t_0 | t_{-1} , t_{+1})}$ is the posterior probability given evidence (which makes sens), $P(t_0)$ is the prior probability of the middle frame (an output of the model), $P{(t_{-1} , t_{+1})}$ is somehow a combination of the output of the model for neighboring frames (I don't know if it is right though, it should be a normalization factor and I don't know if they are the same things). I have the most severe issue with the factor $P{(t_{-1} , t_{+1} | t_0)}$ - I understand it as the probability of distribution of neighboring frames given the probability distribution of the middle frame. But I don't want the data to flow this way. I want to tread the neighboring frames as granted whether the middle frame is valid or not.

Any help is highly appreciated.

Edit:
I oversimplified the problem in order to explain it in a clear way. I am not looking for explanation why my model behaves like that (because in reality the number of classes > 10, and therefore a small change in distribution (attributed to noise etc) can shift the maximum to a different class, and I want to use the neighboring distribution to shift it back, UNLESS the model strongly "believes" that the distribution is correct).

Answer 1

I am assuming that you have labeled training data from which you could estimate, for instance, the probability that if there is a car in frame $t$ then there would also be a car in frames $t-1$ and $t+1$?

It's not entirely clear to me why you would "want to treat the neighboring frames as granted whether the middle frame is valid or not". After all, your procedure should better work on all frames in a consistent manner. If you treat $t-1$ and $t+1$ as fixed and move from frame to frame, then you'd have to treat all frames as fixed. But perhaps I am misunderstanding you on that point.

Another note: using a (Markov) model for $P(C_{t+1}\vert C_{t})$ is probably more convenient than your middle frame variant.

That all being said, I'm not at all an expert in neural networks. If you want to tackle this problem in a somewhat pincipled manner, then you have to look into time-series modeling.

Answer 2

Your first two approaches seem quite sensible, you simply use some kind of smoothing to get rid of outliers.

IIUC, the difference between the first two approaches is simply that in the first one you add classwise probabilities (over a time window) and in the second approach you multiply class probabilities (over a window); in both cases, you normalize in a second step.

The third step: you should use the Bayes theorem only if it makes your life easier, i.e. if you can replace probabilities that you don't know with those that you do know. This doesn't seem to be the case here, though.

If you are not satisfied with the results of your first or second approach, I would suggest looking into other possibilities for smoothing in time series, e.g. Kalman filter. You might also want to think about varying the time window size.