What methods of forecasting should I be looking at to forecast sales?

Question

I am wanting to forecast sales of different products within a business.

I have a good background in mathematics (but mainly focused on analysis, group theory, algebra etc. as opposed to statistics).

I also have some experience in Python + related libraries (pandas, numpy, tf, keras) so anything that could use that would be nice.

The information I have is like:

• Previous sales (including when and how much)
• number of customers that are eligible to buy the product
• how big the customer is
• etc...

I do have access to a lot of potentially related information that could be used but I don't know what to choose.

I have considered doing maybe some kind of seasonal time series (as I suspect there is some seasonality in some products) like ARIMA but I am not sure at all.

Also I want to make it quite simple to begin with I don't want to get carried away with fancy statistical methods that are maybe a) don't understand or b) are not even applicable in my scenario.

I am wanting suggestions of areas/methods/techniques that I could look into that may prove useful.

I don't have terrible amounts of data (on average I would say 3 years of data with 12 points per year [sales in month]) so presumably that would make a difference.

Any advice is appreciated and if clarification is needed let me know.

Thanks.

Answer 1

Simple Models

If you are looking for simple models I would test these:

1. Linear Regression
2. Exponential Smoothing with seasonality
3. Seasonal ARIMA models
4. Rule of thumb methods: seasonal naive method, seasonal mean, simple mean.

These models should help you in assessing a baseline for the model performance. I believe models 1-2-3 are already implemented in the statsmodels library. Depending on how granular your forecast needs to be, you might want to perform a hierarchical forecast. I've used a top-down hierarchical approach for a client (Fashion Retail) that needed forecasts for both product categories and single skus inside the category. Moreover, these models have the advantage of being more transparent and interpretable compared to machine learning models. If your model produces crazy non-sensical predictions, it's much easier to debug them.

Machine Learning Algorithms

If you don't have constraints for model transparency/interpretability, you can employ some Machine Learning algorithms, such as:

• Gradient Boosted Tree Methods (lightgbm, catboost, xgboost, sklearn libraries): these methods don't need a ton of data to be effective, so they should work with monthly observations. It took me a while to study the math behind them, but you don't really need to know how they work to get predictions from them. These libraries are also well documented and there are a ton of examples online.
• Neural networks: they have been employed succesfully both in literature and in online competitions. They usually need a lot of data to shine, though. If you never used them before, I'd try the other models first. However if you are not doing this stuff for work, I suggest you check them out because in my opinion they make use of very interesting concepts and ideas.
• Ensemble models: they combine the predictions of different models, for example by using the average or the median of your models' predictions. This can usually improve your forecast, especially if there is low correlation between the prediction errors of the different models. Of course, it is more time consuming to maintain more than one model in production.

Data

• You should avoid including data features that can't be predicted with a reasonable amount of accuracy. For example, if you have information about daily temperature/weather, you can easily see its effect on past sales, but you can't really use weather forecasts for predicting far into the future, because weather forecasts become unreliable after 3 days in the future.
• Promotional activities heavily impact sales. They can also be planned in advance, so if you can get your hands on this data, try to assess if it can be beneficial to your model.
• Inventory/Stock out information: sometimes it can happen that your product is so in demand, that you run out. If you don't have any information about stock-outs, your demand data is "censored", and you have no way of knowing if you didn't sell because people didn't want to buy or because the product wasn't available. Note that there is a distinction between predicting Sales and Demand. Predicting sales is useful for forecasting financial performance, while predicting demand is useful for optimally allocating inventory.
• Holidays and special events. This information is probably not so useful if you have to generate a monthly forecast. It still might be useful if you have important festivities that sometimes fall in one month, and sometimes in the other (e.g. forecasting sales of Easter eggs and colomba cakes in Italy).
• Outlier removal techniques can be useful for improving the staibility (and sometimes error metrics) of your forecast

Additional sources that I found useful (I wrote a thesis on demand forecasting 2 years)

1. https://otexts.com/fpp2/ My favorite book on forecasting (code implementations are in R, but I found equivalent Python tutorials/examples for most models)
2. kaggle.com has a lot of great notebooks written by really smart and skilled people.
3. https://www.youtube.com/watch?v=68ABAU_V8qI Great video that explains the tradeoffs between complexity and predictive power of a model. It also has some neat "tricks" that can be used in forecasting (although not all are applicable for monthly time series).

Tips:

• Make sure that your model/feature engineering choices are aligned with the granularity and forecasting horizon of the problem at hand.
• if you use Python and pandas to handle your data, the resample function is a great tool. It allows you to quickly aggregate data in periods of specified length (for example, weekly, monthly, quarterly) with just one line of code.
• Transforming the target variable before fitting the model can be useful. For example, if you try to estimate the variable y = log(Sales) and then do an inverse transformation, you can ensure that your predictions are positive.

Answer 2

36 months of data can often be useful in forming a model. Initially, you might want to develop arima type model that incorporates trends, level shifts, and memory while dealing with unusual values.

Secondly, you might investigate adding user-suggested causal variables to your model. These models are an extension and are often referred to as Dynamic Regression OR Transfer Functions or SARMAX models .. https://autobox.com/pdfs/SARMAX.pdf .

Search on SE to find out more about these approaches. You might also post an actual data set and ask for some help.

Answer 3

I would start by looking through the excellent free online book Forecasting: Principles and Practice (2nd ed.) by Athanasopoulos & Hyndman. It leverages the equally excellent forecast package for R, which allows you to automatically build ARIMA or Exponential Smoothing models and forecasts very easily.

With only 36 data points, there is really not much to work with, except for very blatant influences (for which you could look to the xreg parameter for forecast::auto.arima()), so if you want to get more fancy, you would probably have to invest quite some domain knowledge.