1
$\begingroup$

If I have count data, can look 'backwards' at my data, and am trying to predict count for a future window of time, is using Poisson appropriate?

a more concrete example:

  • I have restaurant order data that's like this distribution. enter image description here
  • If I have data of how many orders were placed in the past 30 mins (order rate as well), can Poisson help me model and predict the orders for the next 30mins?

Thanks all,

  • First time Poisson-er.

(bonus points and a place in my heart if you know how to do this in Python with statsmodels)

$\endgroup$
  • $\begingroup$ How much data do you have? (days? weeks? years?) -- Being restaurant data of some kind, presumably there are day-of-week effects, and "holiday" effects, and probably annual seasonal on top of a strong within-day effect. Depending on what the data consist of there may be other effects as well. $\endgroup$ – Glen_b -Reinstate Monica Oct 7 '15 at 22:59
1
$\begingroup$

For restaurant data, it seems strange to only work with one day's data, as there will certainly be important effects based on time of day, day of week, etc., which cannot be properly modeled with just one day's data. Nevertheless, if that is truly what you want to do, then one approach is to model this as a Cox process, which essentially means that orders arrive according to an inhomogeneous Poisson process with intensity $\lambda(t)$, where $\lambda$ itself is a stochastic process. In other words, roughly speaking, $\lambda(t)$ represents the expected rate at which orders are arriving at time $t$.

Given data up through time $t$, you then need some way to estimate $\lambda(t)$. This can be done using the kernel method: you choose some function $f(x)$ with $\int_0^\infty f(x)\ dx=1$ and define $$\hat\lambda(t) = \sum_{0\leq x\leq t} f(t-x)\cdot\text{number of events at time x}$$ The sum here is well-defined, since the terms are zero for all but finitely many $x$. In the simplest case, you could just take $f(x)=1$ for $0\leq x\leq 1$ and $f(x)=0$ elsewhere. In this case, the estimator simply counts the number of events which have occurred in the past hour (assuming $t$ is measured in hours), and uses this for the estimate for the rate at which events are currently occurring. Alternatively, to avoid unnecessary discontinuities in the estimator as orders slide out of the kernel's "window", you could choose $f(x)$ to be a continuous, decreasing function (so that orders further back in time are given less weight). Then, you can model the number of orders which will come in over the next half hour as a Poisson random variable with mean $0.5\hat\lambda(t)$.

The main issue here is how $f(x)$ should be chosen. Just choosing $f(x)$ arbitrarily (as in the last paragraph) is not likely to yield good results. One method is to choose a parametric family of functions (with a scale parameter, for instance) and then use numerical optimization to find the best-performing estimator within this family, based on the past data.

$\endgroup$
0
$\begingroup$

If you bin in 30 minute intervals, your count should be large enough that a Gaussian should approximate the distribution of orders per half-hour. How many days do you have order history for? If it's lots, I would fit a gaussian to the distribution of each half-hour through history to find the expected number of orders and standard deviation for each bin.

$\endgroup$
  • $\begingroup$ only one day. So a poisson regression model that predicts 'order count for next 30 mins' based off predictors like hour of the day, zipcode, and using 'order count for last 30mins' is no good? Could you elaborate on your gaussian answer? $\endgroup$ – SpicyClubSauce Oct 7 '15 at 23:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.