I am facing the following problem: I have a training sample and estimate a model on that training sample. My model is simply OLS: $y_t = a + \beta x_t + \varepsilon_t$. The model is estimated on points in set $t\in T$. The training sample contains well behaved data. When forecasting with this model out of sample, there may be points that are poorly measured and thus take on extreme values. I would like to prevent my model from forecasting extreme output values at times when poorly measured points occur. Thus for points $t \notin T$, I would like my coefficients $(\alpha, \beta)$ to be less sensitive to extremes. I think the appropriate thing is to transform the data in some way (maybe through $ln$)? Maybe Box-Cox?

Let me illustrate. Imagine you have a sensor which functions normally 99.9% of the time but .1% of the time generates a random extreme value that has nothing to do with the measurement. Unless your training set includes that point, you are unable to tailor a model around it. However, you would like not to generate an extreme prediction out of sample when that .1% occurs.

I would like to know what the standard techniques are for dealing with this problem. Please provide some references as well if possible.

  • $\begingroup$ What values are you getting for $\alpha$ and $\beta$? $\endgroup$ Sep 24 '13 at 20:02
  • $\begingroup$ @RobertSmith: I'm not sure I understand your question. Say i get $\hat{\alpha}=0$ and $\hat{\beta}=1$ ... I'm not sure how this helps us though. The issue is that in out-of-sample, I maybe get $x_t$ that is poorly measured (as in 10000 for example). I don't want the model to predict that $\hat{y_t} = 10000$ as well since I know that true $x$ could have been close to 1, for example. $\endgroup$
    – Alex
    Sep 24 '13 at 20:13
  • $\begingroup$ I asked because I wanted to know whether your model was overfitting or not. You say you get a large $y_{t}$ for a large $x_{t}$, so I imagine you have a regression with a really steep line. If the training data is representative of the test data and the outliers are the only issue here, you could preprocess your data with an outlier detector algorithm in order to avoid those poorly measured values. Another suggestion is to add artificial outliers to the training set, however, this will increase the error for most values that, I presume, are already being predicted correctly. $\endgroup$ Sep 24 '13 at 20:31
  • $\begingroup$ @RobertSmith: Thank you for clarifying. Overfitting isn't the issue... it's detecting when a new $x$ has arrived that should be thrown out or shrunk to the mean rather than used. I find your idea of adding outliers to the training data interesting. One still needs to somehow "shrink" these training outliers, hence my interest in Box-Cox transform or some variant. $\endgroup$
    – Alex
    Sep 24 '13 at 20:36
  • $\begingroup$ If the difference between "good" values and "bad" values is very noticeable (1 vs 10000), I think the first suggestion is easier and better, particularly if you're doing well predicting "good" values. $\endgroup$ Sep 24 '13 at 20:46

There are various techniques, starting from non-parametric (such as Theil-Sen estimator) to various optimization techniques with different penalization of residuals. Unfortunately, different estimators behave differently, and only modelling of your data and outliers may help to choose the right one.

  • 2
    $\begingroup$ "and only modelling of your data and outliers may help to choose the right one." --> Regarding this, the RANSAC (Random sample consensus, en.wikipedia.org/wiki/Random_sample_consensus) could be used if you don't have a priori knowledge on what is an outlier and what not. $\endgroup$
    – resnet
    Oct 21 '19 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.