Price optimization using forecasting with ML

Question

I'm thinking about a good method to do discount/price optimization on the SKU level in an e-commerce store. I have enough historical time series data on sales, daily discounts, product views, etc. and stationary data to clearly distinguish products (colors, sizes, brands, etc.). Right now what I think could work is:

• Developing a sales forecasting model on the SKU level, where the target is the sum of next week's sales (bottom-up approach).
• Create an optimizing algorithm to find out the discount prices that give the highest margin on sales.

I'm in the first part, experimenting with various features, models, hyperparameters for the last week and it seems to be a dead-end for me. The target is highly skewed, meaning more weeks with zero sales for a lot of products. The data is converted to a regression problem, where I introduced different lag features for specific time-series features. The training data is constructed in a way (moving window validation) that no knowledge leaking is introduced. When I test my model on a holdout set, the model does an average job on the sum of sales (catching the trend), but on the individual SKU level, it looks really bad.

My model is a simple MLP model, not over-tweaked with just 3 layers, batch-normalization, l2 regularization, and drop-outs. The validation loss was swinging a bit during the training, but after raising the mini-batch size it seems to be solved.

Do you think it is a good way to solve this problem or should I re-construct my architecture? I wonder if there are other proven ways for SKU level price optimization in an e-com scenario. Any ideas, inputs would be highly appreciated.

Answer 1

Do you think it is a good way to solve this problem or should I re-construct my architecture? I wonder if there are other proven ways for SKU level price optimization in an e-com scenario. Any ideas, inputs would be highly appreciated.

In theory, you can use an MLP for just about anything (thanks to the universal approximation theorem) so your approach is technically correct. In practice, using an MLP (or any Neural Network) for this type of problem is very challenging for several reasons:

• You say "Developing a sales forecasting model on the SKU level, where the target is the sum of next week's sales (bottom-up approach)." --- you need to be careful with words here, "Bottom up approach" has a very specific meaning in the demand forecasting context. Specifically it refers to generating forecasts for each individual SKU separately and then summing the forecasts up the hierarchy to obtain class, category, department, etc...forecasts,as opposed to modeling the time series directly at each of those levels. It is part of the more general framework of hierarchical forecasting. I've never heard of summing individual sales per week as "bottom up", although it might be referred to that way in a temporal hierarchy context. Either way, I doubt that you are generating daily or hourly forecasts and then summing them up per week (which would be bottom up forecasting across a temporal hierarchy). More likely, you are summing up actuals per week, and then generating weekly forecasts, which is just the standard practice in Fashion demand forecasting.
• The aforementioned hierarchical forecasting is the "textbook" approach to demand forecasting for a large number of SKUs in most retail and supply chain contexts. You should try that before going down the path of Neural Networks, which are tricky to implement for forecasting (even if they do have some advantages, sometimes, not always).
• If you insist on using neural networks (again, not for the faint of heart), then try LSTMs and seq2seq models, which are better adapted to forecasting problems than MLP.
• Also you say that you have a lot of zeros in your history. These are what are called in the retail demand forecasting world as "slow movers" or an intermittent demand forecasting problem. Slow movers present a whole separate set of challenges from a forecasting perspective and constitute an entire topic of research in and of itself. Your main takeaway should be that you should look into properly modeling the full distribution of your sales, since it is almost impossible to get an accurate point forecast for slow movers. Or at the very least, make sure that you are making the proper assumptions about the demand distribution (Poisson or Negbin, as opposed to the Normal distribution assumption which is common in standard time series models), and that your optimization method is calibrated accordingly.
• Everything above can be done with Neural Networks, but not with a simple MLP, and whether the results will be good or not will depend on your data, and you will face challenges explaining the forecasts to your business stakeholders and also in understanding any causal relationships that you want to use later for the optimization part of your challenge. So definitely, try hierarchical forecasting and the various models for intermittent demand (Croston's, TSB, etc...) first, it will make your life easier.

Answer 2

Id only add to the previous answer that in practice many times you can start by optimizing prices for a subset of products that have regularly enough sales to work with and still get an improvement in profit.

The following paper develops a categorization method of demand.

Syntetos, M & Boylan, John & Croston, JD. (2005). On the categorization of demand patterns. Journal of the Operational Research Society. 56. 10.1057/palgrave.jors.2601841.

You can also see a case study using this type of categorization here.

Regards!