Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I'm not a statistician, so I hope this question makes sense. I'm an EMT and we have patients known to be "frequent flyers". Example, we have a diabetic that we have seen 100+ times in 3 years. I have looked over his pattern of calls and it looks like they follow an exponential distribution, which makes sense to me. However, some times we will see him in clusters. Medically, this does make sense too. If he has a fever he will burn through his glucose more quickly and he may deviate from his baseline "lambda".

I want to test the hypothesis that it is possible to 1) identify a new patient as a "frequent flyer" when N calls approximate an exponential distribution. and 2) identify when an established FF deviates from a baseline (indicating some change in patient status)

A common FF might be that grandma starts falling out of bed more frequently, so this process might not be stationary.

Basically, I think I'm asking what statistical test do I need to take a set of data and show that it is exponentially distributed? I've looked at Fisher Exact, chi Square, and K-S. Also is there a way to determine a "good" value for N? In my case N is the number of times I see the patient, so I would like that to be small if possible, like under 10 so I can flag them for additional non-EMS interventions as soon as possible.

share|improve this question
When you say it looks like it follows an exponential distribution, are you referring to the time between visits? I may be misreading your question, but it seems like you're saying the number of visits follows an exponential distribution. Typically, this would be modelled as a Poisson distribution, and it is well known that the time between Poisson observations is exponential. Also, for testing the hypothesis of whether it follows a Poisson distribution, it would be necessary to have a null distribution, i.e. what does the pattern of "infrequent flyers" look like? – Gschneider May 18 '12 at 19:28
Yes I did mean that the time between visits. When I binned and graphed them it looked like an exponential distribution. – user938512 May 18 '12 at 22:29

2 Answers

up vote 3 down vote accepted

There is a special test of goodness of fit that is designed to have power to detect departures from the exponential. It is called Lillefors test for exponentiality. You should be able to find it through Google. It could be that what you have is not a simple Poisson process but two or more Poisson processes acting over certain time periods but changing over time. Models that might fit your data go by names such as inhomogeneous Poisson process and compound Poisson process. If you Google Lillefors test you will find two goodness of fit tests that go by his name. One is a modification of the Kolmogorov -Smirnov test that takes account for estimating the mean and variance for the normal cdf and the other is designed to reject an exponential distribution. The second one is the one I am recommending.

share|improve this answer
Good answer, Michael (+1). On an unrelated note, if I'm not mistaken, I think I had your son as a student about a year ago. Small world... – Gschneider May 20 '12 at 15:21
Could be. My son Nick is an undergraduate student at OSU majoring in actuarial science and has taken several stat courses. He graduates in a few weeks and is thinking about going for a masters in statistics at OSU. – Michael Chernick May 20 '12 at 15:57
Yep, that's him. Must have been a good student if I still remember him almost 2 years later. Hopefully, we'll be seeing him in the department in the fall. – Gschneider Jun 4 '12 at 19:22
Nick graduates next week but will be applying for grad school in statistics at OSU. – Michael Chernick Jun 4 '12 at 20:01
@MichaelChernick, I like you, so I'm going to pretend you said your son was applying for grad school at the University of Michigan :) – Macro Jun 4 '12 at 22:06
show 4 more comments

Great question!! If I understood well, for "frequent flyers" (FF) the time interval between visits in your hospital follows a exponential distribution with a short half-life parameter ($\theta$).

To me it looks like the visits of the non FF patients also follows an exponential distribution, but with a much larger $\theta$ (correct me if you have evidence against this). The most straightforward model would be to assume that there exists a cut-off value $\theta_0$, (e.g. 1 month) such that for a FF $\theta < \theta_0$. And we now need to set up a statistical framework to test this.

Let us assume for now that you manage to get a good estimate of $\theta_0$ (more on this at the end of the post). Like @Gschneider pointed out, the number of visits in a time interval $t$ will be Poisson. So the probability that a non FF visits your hospital $n$ times or more in a time $t$ is lower than

$$\sum_{k=n}^{\infty} e^{-t/\theta_0}\frac{(t/\theta_0)^k}{k!}.$$

You can compute this score for every patient, $t$ is the time from first visit, and $n$ is the number of visits minus 1 (do not count the first visit). If it is below 0.05 you can reject the null hypothesis with a risk less than 0.05 and conclude that they are FF.

Now your point 2) is more difficult. Before embarking on advanced methods like the scan statistics, 2-state Hidden Markov Models, or breakpoint detection, I would suggest to keep it simple as a first approach. If the model above gives good results, you could decide on a maximum number of visits you are ready to accept before taking action. Say 3 for the sake of the example.

You can estimate the steady-state $\theta$ of a patient by the mean time between visit, excluding the last two visits. With the oversimplistic assumption that this estimate is perfect, the sum of the last two times between visits, say $t_0$, is distributed as a $\Gamma$ variable with parameters $(2, \theta)$. You can use software like R to compute the value of

$$\int_0^{t_0}\frac{1}{n-2\bar{\theta}^{n-2}}e^{-t/\bar{\theta}}t^{2-1}dt,$$

where $\bar{\theta}$ is the mean time between visits, estimated excluding the last two visits, and $n$ is again the number of visits minus 1. This is the (estimated) probability that the sum of the last two intervals would be lower than $t_0$ for that patient. If it is lower than 0.05, you can reject the null hypothesis and flag the patient.

Now, there is the question of the cut-off half-life parameter $\theta_0$. As an expert in your field, you are the only one to have an idea where the line is between FF and non FF patients, and this is the information you have to provide to the statistician. A way to get to $\theta_0$ would be to estimate the mean time between visit for patients that are certified non FF, ideally through confirmation by other experts.

These approaches are very approximate, but already challenging for a non statistician. My opinion is that this question deserves a better answer, but this requires some time. I would recommend you to setup a collaboration with an expert of time series analysis to use more state-of-the-art tools.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.