# How likely is sample A and sample B is from distribution C? [closed]

Let's say I have a sample A: [0,0,0,1] and another sample B: [2,0,5,10,100,3,2,6]

I would like to know the probability that A and B are both picked from the same population C.

I tried applying a hypothesis test, but it gives me a p value of approx. 0.39 and I think it should be clear that it's very unlikely that both samples are from the same distribution.

• I'm guessing you used a pooled 2-sample t test, which is not a good choice here because sample sizes are small, 100 is a far outlier, and sample variances are hugely different. But your intuition that these data are not likely to have come from the same population is correct. Jun 23, 2019 at 18:51
• As phrased the question (which contains a request for a probability), appears to be framed as a Bayesian problem. I expect that a Bayesian analysis is likely not the OP's intent, but if answers talk about hypothesis tests they should also discuss what question those answer (in place of what the question asks). Jun 24, 2019 at 0:09
• What is the distribution of population C? What hypothesis test did you apply? Jun 24, 2019 at 10:49

You don't say what kind of hypothesis test you used. Doing inference on such small samples as these is always going to be difficult.

However, assuming that these are two independent random samples, a nonparametric Kolmogorov-Smirnov test (in R) does reject the null hypothesis that these two samples were randomly sampled from the same population.

There is a warning message that (on account of the ties), the P-value is not exact, but 0.034 seems sufficiently smaller than 0.05 to say that we can reject at the 5% level.

x1 = c(0,0,0,1)
x2 = c(2,0,5,10,100,3,2,6)
ks.test(x1, x2)

Two-sample Kolmogorov-Smirnov test

data:  x1 and x2
D = 0.875, p-value = 0.0337
alternative hypothesis: two-sided

Warning message:
In ks.test(x1, x2) : cannot compute exact p-value with ties


Similar data without ties gives a 'cleaner' test--rejecting the null hypothesis with no warning messages.

y1 = c(.01, .02, .03, .9)
y2 = c(2,0,5,10,100,3,2.1,6)
ks.test(y1, y2)

Two-sample Kolmogorov-Smirnov test

data:  y1 and y2
D = 0.875, p-value = 0.0202
alternative hypothesis: two-sided


Another possible test is the two-sample Wilcoxon (rank sum) test. Its distribution theory is also somewhat disturbed by ties, but it does find a significant difference between your two samples. Looking just at the P-value, we have:

wilcox.test(x1,x2)\$p.val
 0.02434338
Warning message:
In wilcox.test.default(x1, x2) :
cannot compute exact p-value with ties

• You are assuming that the samples A and B are a sequence of IID random variables. Is that clear from the question? It wasn't for me.
– John
Jun 24, 2019 at 4:21
• @John: Yes, explicitly stated now. (a) If not random samples from two distributions, I'm not sure how to interpret the question meaningfully. (b) The P-value quoted in the Question is that of a pooled 2-sample t test, which I took (perhaps rashly) as some evidence of the intent of the question. Jun 24, 2019 at 4:32
• Hi, thanks this solution fits the intended use case. Jun 24, 2019 at 19:31