How are you dealing with the coronavirus "event" in your machine learning models?
Let's say you used to predict the number of sales each month. The virus affected your results last year and it will affect for at least a couple of months. So your model, I guess, is missing a lot. What are some approaches we can use to fix our models?
We do forecasting for retail: supermarkets, drugstores etc. We add predictors to explain our sales time series, specifically different predictors for different phases of the lockdowns.
On the one hand, that will cleanse the time series, so we don't misinterpret higher sales of cat litter boxes as a seasonal effect that will recur next year. On the other hand, this allows us to forecast the possible effect of new rounds of lockdowns. In addition, we remove periods of censored zero sales due to stockouts, like the empty toilet paper shelves in my local supermarket:
Of course, there is a bit of an art to this, or rather a subjective dimension. Which part of COVID-19 influenced sales is a one-off event, and which part reflects a "new normal"? Depending on this, you will want to let your predictors run out, or keep them for the foreseeable future, essentially a "structural break" type of thing.
Shameless piece of self-promotion: I actually briefly mentioned this at a webinar I gave last Friday. Here is the recording. My short remarks on COVID start at 15:45, right after the earthquakes. And yes, I do go into cat litter boxes.
That's an interesting question and I'm sure that there are dozens of different approaches. This is a "there is no wrong answer" type of question.
From my perspective, I've been dealing with the Covid-19 when I think about revenue, which is almost as seeing the sales (as shown by Stephan), but not entirely, since there are things like price involved. These are the strategies I'm working on it and please let me know if anyone see a problem in any of them:
You could also try to find what is the impact of the lockdowns in your country, maybe running a diff-in-diff if the strategy is different for each neighbourhood or city. Then, you would apply this change to fix the predictions your model is making.
Honestly, I'm still figuring this out. This is what I can contribute for now, hopefully, I can bring new insights soon.
I work for a large airline in the revenue and pricing area where we forecast revenue and bookings and other things. We tried 2 approaches. One was effectively scaling from pre-covid data. We tried predicting a lower quantile instead of the mean and having a dynamic scaling factor from 2019. This worked ok. What has worked best for me is just dropping all the pre-covid data and starting from around May. We add in a feature of 'days since the massive covid drop' and that's been very effective.
Since airline revenues dropped by > 90% in some cases when covid started we basically had no ability to adjust since everything was wiped out. Plus there were so many last minute schedule changes and policies that we struggled to incorporate those quickly enough. Our current accuracy is near where it was pre-covid for many models but it varies depending on the approach and problem type.
For the very beginning of covid when we had no data or idea what was happening, I actually just built a Markov Chain model showing how erratic bookings and cancellations were when our directors were asking for forecasts. It opened their eyes to the amount of uncertainty and variance that was present in the pre and post covid environment.
I hope I can tell some interesting stories here just as the others:
I worked on a project for a pharmaceutical company, which wanted to have their dementia and cold cough products to be modeled, especially during coronavirus/COVID-19 infection. For the dementia product, we already saw a decreasing trend that were already active before the crisis. In the end we were 10% past the original sales outcome in our sales forecast, but COVID-19 not really influenced the use of dementia. Although these dementia products are normally also taken in case of sudden deafness (caused by stress) we saw no clear sign of COVID-19, or in other words unusual product specifics that may lead to a higher consumption due to stressful times during COVID-19 and homeschooling.
Thus, the negative trend in the year before was stronger than anticipated, but that was all. We decided to not incorporate any COVID-19 effects and it seemed right. Due to product specifics, we thought it was not correlated so high with COVID-19 lockdowns.
In case of the cold/cough product, you can imagine two scenarios:
We measured effects of coronavirus with dummies, and we already saw unusual activity in January, but I had not enough data to go on much more. The dummies aggregated to 3.000.000 € in sales. No one could have expected what happened:
Hell, after the modeling, the client had a target mismatch of about 50%, but not down; it was going up in early 2020. You can see that the time between January and March 2020 people got crazy on stockpiling. Man, every month they had 2/3 more this aggregated to nearly over 12,000,000!!!
For all three month in average, if we calculated normal seasonal sales we were under 10% in range of our target forecast in sales, but with coronavirus, phew.... And if you remember when coronavirus COVID-19 broke loose, people hoarded toilet paper like Stephan already pointed out, but hell they stockpiled everything that could help against symptoms.
After that I calculated several scenarios for, in Germany we would say: rumdümpeln von Infektionszahlen, by which I mean a shilly-shally of infection cases raising a little bit then going down and again the same thing little wave, up and down.
The other scenario was a lockdown that was lifted right before Christmas eve so that the retailers should profit from late Christmas business. I linked sales to the amount of infected cases. I relatively naively forecasted infected cases based on a Gaussian distribution for sales and infected cases, from week to week, where the top of the Gaussian distribution were already a negative sales impact, and showed a high infection of roughly a few thousand, as our highest number of infected cases up to there were solely 6,000 in Germany. That resulted in very high negative sales during late October till December.
Well, it seems that this modeling was correct in term of lockdown prediction, but as I now leave my company I will never know, how 'low' the sales really were in late 2020. Maybe someday someone will tell me if my amount of negative sales and my big lockdown scenario pointed in the correct direction.
I hope this was entertaining.
Other answers here give some good advice. However, I just wanted to add that economic models should usually be able to incorporate generic "shocks" that affect one or more of the variables of interest. How you incorporat this really depends on what you are interested in explaining, but often the "shocks" will be treated as exogenous events that occur at random times. (Of course, if your goal is to predict the onset of shocks from related observations, you will need to treat them endogenously.)
In the context of a model for sales, it would be usual tfor economic models to incorporate allowance for exogenous "shocks" that can reduce (or increase) the demand for the product at any given price --- i.e., move the "demand curve" up or down. The specifics of how you choose to model this largely depends on whether you want to treat the shock as exogenous or try to explain its onset in terms of other observable things in the model. If you are happy to treat is as exogenous then a simple "random shock" model, where the shock dissipates over time, should give you a starting point.
