2
$\begingroup$

I am trying to simulate a system that will predict Covid 19 weekly incidence rates based on previous health records + several environmental conditions. The system would use all the data available from Dec 2019 up to today to predict the following week's incidence rate and so on and so forth. I am using a SARIMA model to predict those values using statsmodels.SARIMAX and pmdarima.auto_arima. To test the system, I am currently using data from 2019-12 to 2021-02 as my training set and data from 2021-03 to 2021-06 as my test set.

Here is the thing, if I use the whole train set to predict the whole three months in one go (2021-03 to 2021-06) I get fairly good results, however, if I simulate a system where I predict weekly data and then add the predicted value to my train set to predict the following week... in the end, I get a very flat three months predictions.

How could that be?

$\endgroup$
3
  • 1
    $\begingroup$ Adding predictions to the training set is a bad idea. $\endgroup$ Commented Aug 21, 2021 at 12:35
  • $\begingroup$ Can you explain? Would love to know. $\endgroup$
    – Fj86
    Commented Aug 21, 2021 at 12:43
  • 2
    $\begingroup$ I do not have a comprehensive short answer, but note that a model is then trained on data + its own output. I do not see how the latter could help. It would just make the model appear to fit the data better, as it is fitting its own output. $\endgroup$ Commented Aug 21, 2021 at 13:05

1 Answer 1

11
$\begingroup$

Including a graph of your predictions versus the test set might be a good idea.

However, I suspect that your analysis might not yield the desired results for a few reasons:

  1. It is not definitive as to whether COVID-19 follows a seasonal pattern - i.e. spikes at certain parts of the year. There is no way to know for sure with less than two years of data. Therefore, a seasonal factor cannot be known definitively.

  2. Leading on from point 1 - this might be why you are getting a flat line for the three months of predictions. There is a lot of volatility in the data and ARIMA cannot predict such volatility.

  3. Additionally, ARIMA relies on past data to predict the future. Using such a model is not a good idea if there are interventions that can affect future trends. For instance, a significant portion of people globally are now vaccinated against COVID-19, but this was not the case during the period across your training set. This means that past data will not incorporate such information that will be reflected in future data.

  4. As was mentioned in the comments, you should refrain from adding predictions to your training set. This influences the predictions that will be made in subsequent time periods. If the prediction added to the training set was inaccurate, then subsequent predictions will also be inaccurate. The training set and test set also need to be kept entirely separate to avoid data leakage - where the model gives the illusion that it is predicting well but in fact it is because data from the test set is influencing the model.

I don't know if your end goal is simply to predict COVID-19 cases or if this is part of a wider analysis, but you might wish to reconsider your approach given the points above.

As a disclaimer, I am not a health specialist and none of the above is meant to be of any material advice regarding COVID-19 specifically. This question is simply answered from the point of view of explaining time series principles.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.