# Detecting if samples belong to a given distribution

I am a networking person and I am trying to use statistical methods for a problem I am facing, so I would appreciate any help or pointers.

I have users that access the medium, but in case 2 or more try to access it, they are forced to wait before they retry. The assigned wait times are random and follow an exponential distribution.

I need to figure out if they break the rule and start using a more favorable distribution (it doesn't matter which distribution: I only care to detect when they deviate from the exponential). I have done some readings and came up with CUSUM method, but I am not sure if there are better approaches.

• So you want to test exponential against the alternative of something that's typically smaller (but of unknown shape)? Does your exponential have a known mean? – Glen_b Feb 26 '15 at 19:58
• Probably I could determine the mean using a training set. I have a set of incoming samples and I would like to decide whether the series of incoming samples is following the pre-set set of rules. – user20150316 Feb 26 '15 at 21:04
• I'm not asking you to specify a mean, I'm asking if you have one. If you're basing it on a training set that's a sample value rather than a population value and should be treated as one in the test. – Glen_b Feb 26 '15 at 21:09
• Than, no it doesn't have a known mean. – user20150316 Feb 26 '15 at 21:51

## 2 Answers

If you seek formal hypothesis tests:

a) If you want to test stochastic dominance of the training set over the new sample, you could use a one-sided two-sample Kolmogorov-Smirnov test (or perhaps a Mann-Whitney test)

b) If you're more interested in them shifting the mean, you might use a permutation test, or

c) with the assumption of exponentiality under the null, you could construct an $F$-test based on the ratio of means of the training set and the new sample ... again, in either case, one-sided tests can be done, since you seem to have only a one sided alternative that's of interest.

If you just want a diagnostic:

a) You could plot overlaid quantile plots (vs expected exponential quantiles) for the two (training set and new set), though there are other ways -

b) you could plot both ECDFs, for example

I believe that Anastasia wants to calculate the departure from the Exponential distribution as new data arrives. Thus in essence, she can perform a changepoint test to decide if the recent data obeys the same distribution as previously seen data.

There are several options here, if you are sure about the Exponential assumption then you could use a likelihood ratio test statistic. The downside with this is that you are also assuming an Exponential distribution for the new data but just with a different rate parameter. This may work in practice and give you the answers you want but you would have to try it.

What might be more beneficial is to use a nonparametric test statistic such as a Kolmogorov-Smirnov test as described by Glen_b. However, I would adovate using this in a changepoint setting so that as a new data point arrives, you see if a change has occurred recently.

Both these methods are available in the cpm R package. Unfortunately due to not passing some CRAN checks, this has been archived here. I have used the pacakge for a while and can vouch that the methods are correct, the package has just failed a CRAN check on the Solaris operating system which the package author is finding awkward to correct. You can install it manually by downloading the source from the archive. I don't believe that there are any other changepoint packages on CRAN that are capable of detecting changepoints from a data stream.

• Yes, I was initially thinking of change point detection, but I was confused with the fact that I have 2 exponential distributions (I don't know which distribution I will have and I don't care: I just care to detect that it is not exponential). Is the R function called change point detection? – user20150316 Feb 27 '15 at 17:45
• For the cpm package you need the detectChangePoint function or detectChangePointBatch if you are working offline (e.g. for testing). – adunaic Mar 4 '15 at 9:16