0
$\begingroup$

I have a feeling this is a somewhat common problem but reading about tests for normal distributions hasn't really helped me as their assumptions didn't seem to fit mine.

Consider the following case:

  • I have data (a lot) that is somewhat normally distributed but quite leptokurtic.
  • I collect new data samples and want to test if the new data is from the same distribution.

What test should I use here? To give some context the testing would be done online (e.g. repeated after 10, 20 etc. samples) and I am interested to find out how many samples I need to reject the null hypothesis (where appropriate).

Edit: Alternatively, is there some way to forgo the distribution/goodness-of-fit testing and evaluate new incoming data according to its likelihood? I.e. data point 1 is in the 90-percentile of the distribution, point 2 in the 95% and so on. Eventually, I want a test to tell me that the behaviour is likely according to the previously collected data or not.

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Please note that the term 'likelihood' has a specific, technical sense in statistics that it doesn't seem like you intend here. Another form of phrasing might head off that potential confusion, though it might suffice to simply note that you don't intend that sense. $\endgroup$ – Glen_b Jul 28 '14 at 0:39
1
$\begingroup$

you can use the Kolmogorov Smirnof test: Consider two samples ${x}$ and ${y}$ and let $F_n(x)$ be the empirical distribution function of $x$ and $G_m(y)$ be the one of $y$, where $n$ and $m$ are respectively the sample size of the first and second sample. The statistic is

$D = sup_{x,y} |F_n(x)-G_m(y)|$.

The critical values (of the Kolmogorov distribution) are tabulated.

In compute the critical value you have to take in consideration that you want to do many sequential test and then the critical values must be adjusted. I am not an expert in that field but you probably can find something on the internet. (the key words are "sequential test", "adjusted p-value"..)

EDIT: I changed the notation accordingly to the suggestion of Russ Lenth

Improve this answer
$\endgroup$
5
  • $\begingroup$ I don't think the goal is to test for normality. It's to see if the new data have the same distribution as the old data - which is not normal. $\endgroup$ – Russ Lenth Jul 27 '14 at 19:28
  • $\begingroup$ Sorry, i read the question too quickly :).I made some change $\endgroup$ – niandra82 Jul 27 '14 at 19:39
  • $\begingroup$ @RussLenth To be clear niandra's (now) presented the two-sample Kolmogoro-Smirnov test, which is a test of whether two samples are drawn from the same distribution, as, I think, you are asking for. $\endgroup$ – Alexis Jul 27 '14 at 19:44
  • $\begingroup$ @mandra82 This is better, but I don't like the notation, because you have used $F_n$ to denote two different functions (calling them with different arguments doesn't change the function!), and there are two different sample sizes. I suggest $F_n(x)$ and $G_m(y)$. $\endgroup$ – Russ Lenth Jul 27 '14 at 19:57
  • $\begingroup$ Thanks, when I run the two-sided kstest in Matlab with my data even a subset of that data (unless it's extremely small or extremely large of course) would get rejected. What am I missing there? $\endgroup$ – user160531 Jul 27 '14 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.