Hidden markov model to detect Stock outs in Hourly sales Time series data

Question

I'm currently dealing with time series data about sales of beverages in a supermarket. I have data for each minute, but I'm aggregating by hour since the dataset is already huge with a daily granularity.

I would like to detect stock outs occurring to a product, where a stock out happens when all the items of a given product where sold and the product is not available until refilled. I was suggested to use a Hidden Markov Model, but having little knowledge about it I don't know how to set up the problem, so I ask for help with references and suggestions of the statistical and mathematical assumptions to use. My idea would be something like:

$Y_i$ = hourly time series of sales, $i=1,...,n$,

$Y_i \in Z^+$

$Z_i $ = hourly time series of latent states, $i=1,...,n$

$Z_i \in {0,1}$, with $0$ corresponding to no items available/stock out

Basically I would like to infer the latent state given the observable sequence of sales.

I'm currently trying to find it using depmixS4 library in R:

dep2 <- depmix(resp ~ xreg,nstates=2,family=poisson(),ntimes=length(resp))
hmm2 <- fit(dep2) 
ba2 <- BIC(hmm2)
summary(hmm2)

Thank you in advance

Answer 1

Build a Transfer Function model for hourly sales that takes into account hourly effects , daily effects, weekly effects, monthly effects, holiday effects , day-of-the-month effects, week-of-the-month effects, month-of-the-year effects , current level and trend effects , holiday effects, long-weekend effects , price effects , promotion effects , weather effects et. al. while taking into account any ARIMA effects. This will allow you to identify anomalous behavior via Intervention Detection . If the predicted sales are non-zero for any identified Intervention Point in time and the observed sales are zero then you have identified a potential stockout as the observed zero was significantly different from the expected non-zero .

You might want to look at Seasonality in residuals ACF and PACF to review how you might build the 24 hourly models using empirically identified structure and user specified causal variables .

Hidden Markov Models deal with latent or hidden variables. What I proposed was a way of identifying the latent/hidden variables by appropriately layering in data-suggested effects (hourly,daily etc) in sufficient quantity and form without collapsing the solution. To form these (waiting to be discovered variables) takes aggressive heuristics employing a lot of trial and error . So you might conclude that intelligent time series modelling is just one very aggressive Markov Model reaching into the data to form sufficient structure to render a white noise error process.

I am not an expert in HMM but I would be very interested in a bake-off comparing HMM and Transfer Function Modelling for your data if someone else can come forth to provide a HMM solution.