Demand forecasting: training a model using falsified historical sales as a target instead of actual sales

The framework:

We have different products being sold for a short period of time (maximum 3 weeks) once a year. Some products may have no sales history - new products. Since this is not a classical time series forecasting we decided to use ML (Machine Learning) approach by estimating the future demand based on the history of sales using different features like price, name of a product (text based features), store information, store location, etc. The assumption is that the model will learn complex interactions between all features and will forecast sales for the next year.

Optimisation goal

The forecasted demand should lead to the situation that a predefined percentage (%) of units of each product will be sold in the first 7 days even so a product can be sold in total for maximum 3 weeks. A human expert is responsible for forecasts that should theoretically reach the goal.

Preliminary analysis

The analysis of historical sales showed that the forecasted demand doesn't optimize for percentage sales. Products were sold in any day within 3 weeks interval without peaking at day 7.

Proposed Approach The logic of the proposed approach is the following

1. Use some heuristic that will retrospectively find the demand for a product that should have led to X% of sales.
2. Apply heuristic on all historical sales and create a new "would-be" sales
3. Train a model on a "falsified" sales data

My question to experts: what would be the consequence of this approach?

The analysis of historical sales showed that the forecasted demand doesn't optimize for percentage sales. Products were sold in any day within 3 weeks interval without peaking at day 7.

It sounds like your demand insists on not following the process you have defined. Dastardly reality!

Seriously, it is not really a surprise that your model does not fulfill your expectations. After all, you are (presumably) forecasting either expected total demand, or a quantile of total demand, so your objective function does not care about the temporal dynamics.

The forecasted demand should lead to the situation that a predefined percentage (%) of units of each product will be sold in the first 7 days even so a product can be sold in total for maximum 3 weeks.

Note that you have two conflicting goals here. If your demand is the same every week, then after the first of three weeks you will have seen 33% of total demand. However, if demand is strong in the first week and then falls off (which may in particular happen if you get a reputation of having low service levels - then people will rush to buy in the first week, because they expect you to be out of stock in the second or third week), then you may already have seen 50%, 75% or an even larger share of total demand after the first week is over. So you really need to decide which goal is more important:

• Either you want to have enough stock to last for three weeks with a prespecified probability. Don't aim at always having enough stock for three weeks, because then you will need enormous safety amounts, and have a lot of product left over. 90% or 95% service levels are usually a good thing to aim for.
• Or you want to order enough stock so that you expect to have sold a specified percentage of your stock after one week. As above, depending on the dynamics of demand, that may mean that you systematically run out of stock in the second week, or systematically have leftovers after three weeks.

If you decide to aim for the second way. I would proceed as follows: forecast expected (!) demand for the first week only. Scale this up by the reciprocal of your desired sell-through percentage. (If you want to have sold 25% of your stock after the first week, then your stock should be four times the expected demand of the first week.) You're done!

Note that in this case you should not do quantile forecasts, because you are already setting safety amounts through the scaling of the expectation forecasts.

In any case, before you choose either one of these approaches, I would take a critical look at the requirement of having sold through a given percentage of stock after the first week. Something that more flexibly adapts to the actual course of demand over time is likely more useful. And my personal opinion is that a total service level over your selling seasons makes more sense as a target.

• I need some clarification on forecast **expected** (!) demand for the first week only - what does expected mean in this context? the expected demand based on actual sales for the first week? After adding safety stock on top of a prediction I guess the comparison should be between the ML (forecast + safety stock) against Human Expert (deliver quantity) versus actual sales (within 3 weeks)? – slavakx Mar 5 at 12:14
• "Expected" here means that you would need to forecast the expected value of the future demand distribution, not, say, a 90% quantile. (Because with a high quantile, you would add safety amounts on top of safety amounts.) Your ML method very likely already does this, so you can just use its point forecasts. And yes, after scaling this forecast up, you could compare it to a human expert's total quantile forecast as in the other thread. After all, scaling the first week's forecast is nothing else than a particular way of calculating the quantile forecast, as discussed in the other thread. – Stephan Kolassa Mar 5 at 13:13