I have some broker prices incoming in real-time for several products. Sometimes a broker makes a typo and sends a wrong price, or my parsing engine assigns the price to the wrong product - these are the two ways an outlier can occur.
I wish to remove the outliers, in real-time as they come in. I cannot just look at each time-series of each product individually since they are non-stationary and exhibit jumps as the market changes. My idea was to compare the incoming price not only with the past prices of the product it got assigned to, but also other products in order to see whether:
- Whether the price 'fits' well into the assigned product
- Whether the price fits better into another product with a similar name (probably a parsing error)
- If the new price was assigned to a product but it is quite different in value than recent prices - check with other related products if they all also jumped, then the new price is probably fine as the whole market jumped.
I have these ideas but cannot put them together in terms of what outlier techniques to use. I prefer simpler methods rather than cutting-edge unsupervised ML algos since it is important there is explainability.
Advice is appreciated.