# How can I evaluate smoothed offline predictions for later real-time use without future bias?

I have a system that makes predictions in real time. Whenever the system encounters a positive label, the process must be stopped. This, however, means that noisy predictions, such as the one shown at time 10, is a false positive that should be removed, in favor of the later, more certain predictions.

As I'm evaluating the performance offline on already-happened data, I'm afraid to introduce future bias. Consider, for example, some sort of forward-moving filter. Even in a short timespan, several filters would contain future information, so these are obviously no-go.

How can I smoothe and evaluate in a way that mimics future unseen data?

import matplotlib.pyplot as plt
import numpy as np

p = np.random.normal(0, 0.1, 30)
p[10] = p[10] + 1
p[20:] = p[20:] + 1
p = (p - min(p)) / (max(p) - min(p))

fig, axes = plt.subplots(nrows=2, sharex=True)
axes[0].plot(p)

for start in range(30 - 5):
end = start + 6
c = "red" if start < 10 and end > 10 else "black"
axes[1].plot([start, end], [start, start], color=c, alpha = 0.5)

for ax in axes:
ax.axvspan(0, 20, alpha = 0.1, color="red")
ax.axvspan(20, 30, alpha=0.1, color="green")
ax.axvline(10, color="black", ls=":")

plt.show()