I'm trying to do time series prediction and I'm interested in training on MAE or other custom loss functions. For my problem I'd prefer having errors of {0, 10, 0, 10, 0, 10} as supposed to {5, 5, 5, 5, 5,5} so that I am occasionally hitting i.e. getting error=0. I tried rlm() in R and that saw an improvement versus lm() since I'm guessing it's giving less conservative projections and not worried about missing on outliers, so for that reason I'd like to try more models that use MAE or allow for custom loss function. Linear models have been working best so far, but I'm interested in suggestions on any models that might help. Thanks

  • $\begingroup$ What interval are you calculating when you use the linear model (is it for example, an interval for the mean - the usual regression confidence interval instead of a regression prediction interval? Please be clearer about what you did. $\endgroup$ – Glen_b Jan 21 '17 at 4:58
  • $\begingroup$ I'm not calculating an interval. I'm training a linear model and predicting on data in the future and the resulting prediction ends up just being very close to the mean of the training data. I'm hoping to get predictions that can have more variance to them so have more hits and misses on data in the future rather than just being always close to the mean so was wondering if there are models that could be useful for this. $\endgroup$ – Brian L Jan 21 '17 at 5:07
  • $\begingroup$ It's not at all clear what you mean. Please be considerably more explicit. $\endgroup$ – Glen_b Jan 21 '17 at 7:08
  • 2
    $\begingroup$ @Brian L Thanks -- the example you give helps. What you're getting at there is that your loss function is not well modelled by using an $L^2$ norm. So the problem is not one of increasing the variance but of correctly defining what you want your predictions to do. For example if you want to predict close to the most likely value, don't use minimum mean square error predictions. If you describe your problem in more detail in your question - along with what properties you want your predictions to have (i.e. what you want to do well at) you will be likely to get better advice. $\endgroup$ – Glen_b Jan 22 '17 at 3:06
  • 1
    $\begingroup$ @whuber I'm just wondering if there are any linear models that train on MAE or allow for custom loss function. I think my question may have been confusing because I'm basically just trying to use a different loss function but don't know how to do that with a simple model like a linear model. I tried adding random noise but it makes MAE worse. $\endgroup$ – Brian L Jan 23 '17 at 18:02

Since this is time-series, you may want to try the ETS framework. Some models are linear, others not. The R forecast package has an ets function in which you can optimize via a number of criterion, including MAE and it will pick the best model for your data in that framework.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.