0
$\begingroup$

I have two very large distributions, I did anderson-darling test on these two distributions and the result shows they're from different populations, significantly. Now I want to know which samples in these two distributions are making them different. Is there a way to tell?

To be clearer, here's an example:

library(kSamples)
set.seed(1234)
ad.test(rnorm(100, 0, 0.1), c(rnorm(100, 0, 0.1), rnorm(10,100,1)))

and result shows:

      Anderson-Darling k-sample test.

Number of samples:  2
Sample sizes:  100, 110
Number of ties: 0

Mean of  Anderson-Darling  Criterion: 1
Standard deviation of  Anderson-Darling  Criterion: 0.75453

T.AD = ( Anderson-Darling  Criterion - mean)/sigma

Null Hypothesis: All samples come from a common population.

             AD  T.AD  asympt. P-value
version 1: 4.78 5.009         0.003532
version 2: 4.80 5.030         0.003476

Now if I remove the last 10 samples from the latter distribution:

set.seed(1234)
ad.test(rnorm(100, 0, 0.1), rnorm(100, 0, 0.1))

the result becomes:

 Anderson-Darling k-sample test.

Number of samples:  2
Sample sizes:  100, 100
Number of ties: 0

Mean of  Anderson-Darling  Criterion: 1
Standard deviation of  Anderson-Darling  Criterion: 0.75419

T.AD = ( Anderson-Darling  Criterion - mean)/sigma

Null Hypothesis: All samples come from a common population.

              AD  T.AD  asympt. P-value
version 1: 2.354 1.795          0.05841
version 2: 2.350 1.790          0.05872

Is there a way, given certain p-value (0.003532) to p-value (0.05841), to identify the last ten samples, which make the latter distribution more different than the former one?

And also, why is the p-value of ad.test(rnorm(100, 0, 0.1), rnorm(100, 0, 0.1)) so small (0.05841) by a-d test even if they're actually two distributions drawn from the same population by rnorm?

$\endgroup$
0
$\begingroup$

First of all, removing multiple values from a sample will also change the sample size. This will generally increase p-values, so be careful with that. To answer your question, in glm's it is common to compute the cooks distance statistic. For a sample of size n, this involves creating n copies of your dataset, and removing one value from your dataset in each copy. Subsequently, you determine the statistic of interest (e.g. AD test) and see removing which values has the largest impact. This is technique is essentially a jackknife, which is a member of the bootstrap family.

$\endgroup$
  • $\begingroup$ I see. Basically I can remove each sample and do the same statistic test to see the impact. Then I can rank each sample by the impact of removing them, right? Great idea. Thanks for your help! $\endgroup$ – Yan Nov 30 '16 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.