5
$\begingroup$

I am running ridge regression on time series data for the purposes of prediction. The data is non-normal, highly correlated and prone to fat tails either way (financial data). I am not removing the data because or errors, just because it helps prediction.

I currently standardise and remove anything above $|x|$ = 3, 4, or 5 standard deviations, to improve prediction based on MAE, MSE and adjusted $R$2. However I was wondering if this is a good approach or if clipping or perhaps winsorizing data is generally a preferred method in these cases. Are there a good arguments to be made for any of the method over the others?

Improve this question
$\endgroup$
7
  • 5
    $\begingroup$ If the extreme values are not errors in the data, then you are not helping prediction by removing them - you are merely ignoring data that your model does not explain well. I strongly suggest you retain them in your dataset. Even if you cannot explain them with your model, your estimates of predictive power will be more accurate than if you exclude them. If you think they are biasing your results, you could use a robust approach (which you seem to have attempted, with the use of MAE instead of MSE) $\endgroup$
    – mkt
    Aug 18 '17 at 7:03
  • $\begingroup$ Just a thought: When I cross validate on out of sample I also improve my metrics, I am predicting volatility, so i believe it does tend to bias the result higher or make it less sensitive to small changes of explanatory variables possibly. When you say robust do you mean outlier removal to optimise MAE? $\endgroup$
    – azuric
    Aug 18 '17 at 7:15
  • 1
    $\begingroup$ If you have removed the extreme values from your out-of-sample data, it is not a fair test. As for robust, I do not mean outlier removal. I mean choosing methods that weight extreme values to a lower degree. E.g. MSE will square your residuals, meaning that extreme values will have a large influence on the outcome. Those values will have a much lower influence if you use MAE because of the lack of squaring. There is an entire subfield of 'robust statistics' that deals with problems such as yours, without removing data. $\endgroup$
    – mkt
    Aug 18 '17 at 7:45
  • $\begingroup$ no i keep extreme values in the out of sample $\endgroup$
    – azuric
    Aug 18 '17 at 7:48
  • 1
    $\begingroup$ Yes, that would be an improvement $\endgroup$
    – mkt
    Aug 18 '17 at 8:44
8
$\begingroup$

Turning my comments above into an answer:

If the extreme values are not errors in the data, then you are not helping prediction by removing them. You are instead ignoring data that your model does not explain well. I strongly suggest you retain them in your dataset. Even if you cannot explain them with your model, your estimates of predictive power will be more accurate than if you exclude them. If you think they are biasing your predictions/results, you could use a robust approach (which you seem to have attempted, with the use of MAE instead of MSE).

By robust methods, I mean methods that weight extreme values to a lower degree. E.g. MSE will square your residuals, meaning that extreme values will have a large influence on the outcome. Those values will have a much lower influence if you use MAE because of the lack of squaring. There is an entire subfield of 'robust statistics' that deals with problems such as yours, without removing data. Take a look at the tag.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Adding a comment: in many circumstances, extreme values are real data. For example, most people have pretty low healthcare spending in any one year (a few doctor visits, some prescriptions). Some people get hospitalized, and some of them get hospitalized with a lot of complications or a severe injury. The latter category may look like outliers, but their data are real and they really contribute to the average. If anyone removed outliers from a healthcare spending dataset, I think a lot of people would object unless there was evidence that the claims were made in error or fraudulent. $\endgroup$
    – Weiwen Ng
    Sep 10 '19 at 16:43
  • $\begingroup$ @WeiwenNg Good point. That is a good example of a common, non-normal distribution where extreme points are to be expected. These should not be treated as outliers - the models should be designed with the appropriate distribution in mind. $\endgroup$
    – mkt
    Sep 10 '19 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.