# Will removing outliers improve my predictive model?

I am running ridge regression on time series data for the purposes of prediction. The data is non-normal, highly correlated and prone to fat tails either way (financial data). I am not removing the data because or errors, just because it helps prediction.

I currently standardise and remove anything above $|x|$ = 3, 4, or 5 standard deviations, to improve prediction based on MAE, MSE and adjusted $R$2. However I was wondering if this is a good approach or if clipping or perhaps winsorizing data is generally a preferred method in these cases. Are there a good arguments to be made for any of the method over the others?

• If the extreme values are not errors in the data, then you are not helping prediction by removing them - you are merely ignoring data that your model does not explain well. I strongly suggest you retain them in your dataset. Even if you cannot explain them with your model, your estimates of predictive power will be more accurate than if you exclude them. If you think they are biasing your results, you could use a robust approach (which you seem to have attempted, with the use of MAE instead of MSE)
– mkt
Aug 18 '17 at 7:03
• Just a thought: When I cross validate on out of sample I also improve my metrics, I am predicting volatility, so i believe it does tend to bias the result higher or make it less sensitive to small changes of explanatory variables possibly. When you say robust do you mean outlier removal to optimise MAE? Aug 18 '17 at 7:15
• If you have removed the extreme values from your out-of-sample data, it is not a fair test. As for robust, I do not mean outlier removal. I mean choosing methods that weight extreme values to a lower degree. E.g. MSE will square your residuals, meaning that extreme values will have a large influence on the outcome. Those values will have a much lower influence if you use MAE because of the lack of squaring. There is an entire subfield of 'robust statistics' that deals with problems such as yours, without removing data.
– mkt
Aug 18 '17 at 7:45
• no i keep extreme values in the out of sample Aug 18 '17 at 7:48
• Yes, that would be an improvement
– mkt
Aug 18 '17 at 8:44