# How to forecast for the next year given I have past 3 years data

I have a variable which is "% of a company's customers that participate in certain program" (that the company offers). Given that I have only data for past 3 years (I have month level data too for the variable) how do I forecast the next year's value ?

example - I know 53%, 54%, 52% of our customers completed a program in the past 3 years and need forecast for next year. Say the company has millions of customers. There are few variables (customer level) like a person's age, engagement, 'participation in a survey', 'contacting the company' etc. that may impact the final variable. But, on average all these customer level variables look almost same every year.

What methods (other than some sort of the average of the previous values) can I use to forecast next year variable (so from the past 36 months, I need to forecast next 12 months, and can use any customer level variables).

Methods I am looking for are not some exponential weighted averages or simple ARIMA like ( which are just basically also kind of some averaging of previous values etc.). Are there any other methods other than some sort of weighted averages methods? If possible, I am interested in methods where I can be using the customer level variables as well (I can only use some of the variables like age, "if the completed the program last year" etc. as I am looking to forecast for next year and not prediction).

• Think about what you’re asking. You want to predict the future given a handful of observations of the past. There obviously won’t be a single simple & unambiguous way of doing that. Mar 14, 2021 at 23:47
• @DemetriPananos will the 36 monthly points help to create the next 12 months?
– tjt
Mar 15, 2021 at 0:17

Simple methods applied to your aggregate, like ets() and auto.arima() in the forecast package for R should definitely be your first try. They run automatically with a few lines of code and represent the gold standard in forecasting. See Forecasting: Principles and Practice (2nd ed.) by Athanasopoulos & Hyndman.

Since your percentages are apparently far away from 0% and 100%, you don't need to worry about special methods for .

Beyond that, you could of course also look at customer-level forecasts as to whether this customer will participate next year or not. Run a logistic regression on all of your customers, including the past year's explanatory variables. If you want to get fancy, do a GLMM with a random intercept per customer to model each customer's general propensity to participate. (That will only help you if you don't have a lot of churn, though.) Then predict this out and aggregate the predicted probabilities.

Actually, at this point you have a hierarchical model, with the model on the top given by the time series model and millions of bottom level probabilistic forecasts. So you can use hierarchical reconciliation to put the two together. Take a look at the hierarchical forecasting chapter in the textbook linked above.

However, for your logistical models you will need to include in some way how the monthly aspect comes in. Do people enroll for a full year, and you need to predict whether a given customer will renew, or enroll for the first time, or re-enroll after their participation has lapsed for one month, two months, three months...? If so, you will likely need different models for all these cases and in the prediction step use the one that is appropriate for the specific customer.

Overall, these bottom level models and forecasts will be far more complex to set up and tune (and note that tuning them will lead to some overfitting, so you may be overoptimistic about your models' performance on unseen data) than the time series models. Plus, you say that the aggregate predictors don't change much over time. In this case I suspect that any improvement over the top level aggregate time series model will be modest at best.

• To agree with what StephenKolassa said those are the simplest and in ESM case the most robust ways to do time series. ARIMA makes assumptions that your data may not support -ESM has almost no assumptions people pay attention to. :) 36 data points is probably not enough data if you have seasonality, but you use what you have. The Smooth module of R has more ESM options. Mar 16, 2021 at 22:23