I'm thinking about a good method to do discount/price optimization on the SKU level in an e-commerce store. I have enough historical time series data on sales, daily discounts, product views, etc. and stationary data to clearly distinguish products (colors, sizes, brands, etc.). Right now what I think could work is:
- Developing a sales forecasting model on the SKU level, where the target is the sum of next week's sales (bottom-up approach).
- Create an optimizing algorithm to find out the discount prices that give the highest margin on sales.
I'm in the first part, experimenting with various features, models, hyperparameters for the last week and it seems to be a dead-end for me. The target is highly skewed, meaning more weeks with zero sales for a lot of products. The data is converted to a regression problem, where I introduced different lag features for specific time-series features. The training data is constructed in a way (moving window validation) that no knowledge leaking is introduced. When I test my model on a holdout set, the model does an average job on the sum of sales (catching the trend), but on the individual SKU level, it looks really bad.
My model is a simple MLP model, not over-tweaked with just 3 layers, batch-normalization, l2 regularization, and drop-outs. The validation loss was swinging a bit during the training, but after raising the mini-batch size it seems to be solved.
Do you think it is a good way to solve this problem or should I re-construct my architecture? I wonder if there are other proven ways for SKU level price optimization in an e-com scenario. Any ideas, inputs would be highly appreciated.