4
$\begingroup$

Conformal predictions allow one to add prediction intervals to arbitrary machine learning regression models. For more information see Algorithmic Learning in a Random World.

An assumption of conformal predictions is that the observations are exchangeable, a weaker version of i.i.d. I would like to apply conformal predictions to a model where the input data consists of events that take place during the year. The expectation is that there are some seasonal effects. So these observations are neither exchangeable nor i.i.d.

In the model though we do take into account seasonal attributes like month of year and other indicators. Our assumption is that the errors of the model predictions are more or less i.i.d. through time. Since the conformal predictions hinge on order statistics calculated on known prediction errors, could conformal predictions in this case still work?

For those interested, roughly conformal predictions work on the observation that if one draws a random sample, and then orders the outcome descending then it is possible to assess with which probability a next random observation will for example be smaller than the 8th observation in the original sample.

$\endgroup$
1
  • 1
    $\begingroup$ Could you describe the conformal approach you used? Also, what do you mean about conformal prediction "work"? Do you mean marginal coverage under the exchangeability assumption of the truth? In the review arxiv.org/pdf/2107.07511.pdf, there are some extensions of conformal prediction to distribution shifts due to the time series setting. However, if the seasonal effect is the only issue related to the iid setting, it is possible that the joint exchangeability assumption holds (e.g. same time of the year in the test set and also out-of-sample), but the answer depends on your assumptions $\endgroup$
    – Jim
    Commented Feb 14, 2023 at 22:12

1 Answer 1

2
$\begingroup$

There are conformal prediction approaches that explicitly consider time series analysis. Here is an article that discusses this: Conformal prediction for time series. The R caretForecast and the Python MAPIE packages can implement such methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.