8
$\begingroup$

From a larger sample of tabular data, I have picked certain rows that meet a certain condition (this condition is unrelated to the actual data in the rows).

Now, I want to know if the distribution of this subset that I have created is similar to the distribution of the original, larger sample.

What test(s) can I use for this purpose?

Thanks! I appreciate the help.

Improve this question
$\endgroup$
3
  • $\begingroup$ What does your data look like? Is it discrete or continuous? $\endgroup$
    – soakley
    Commented May 13, 2014 at 18:27
  • $\begingroup$ It's continuos. The data is basically words with sentiment scores (in the range of -5 to 5). $\endgroup$
    – Debi
    Commented May 13, 2014 at 18:31
  • $\begingroup$ What if the data is discrete instead? Can the Kolmogorov-Smirnov test be used in that case? $\endgroup$
    – rezita
    Commented Jun 10, 2017 at 21:39

2 Answers 2

Reset to default
5
$\begingroup$

You could test whether several statistics that are descriptive of a distribution are the same in the subsample and the remaining sample. For example you could conduct tests for:

  • mean difference
  • median difference
  • stochastic dominance
  • different variance
  • shape

While you are at it, since you are interested in similarity, I would also explore tests for equivalence of all such measures (for example, using tost), probably combining inferences from difference and equivalence tests.

Something else you may want to consider: why are you interested in this similarity? The answer to this question may help you decide which, if any, such tests you may like to explore. For example, if you sample size is smallish, you may not have enough power for the Kolmogorov–Smirnov test mentioned by soakley, although you might still have power enough to make inferences about, say, the sample mean. If you are only interested in comparing sample means, that may be OK for your purposes.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi, as a follow up: The tost test (in R) asks for a magnitude of region of similarity. Any idea what this is? $\endgroup$
    – Debi
    Commented May 13, 2014 at 19:00
  • $\begingroup$ Yup: you need to make a decision about how large a difference is meaningful. In other words, the minimum size of difference that you are willing to accept as important. See for more info: stats.stackexchange.com/tags/tost/info (and pay attention to what $\Delta$ is). $\endgroup$
    – Alexis
    Commented May 13, 2014 at 19:36
  • $\begingroup$ is it also possible to check a sample though the population is not known? $\endgroup$
    – Ben
    Commented May 20, 2022 at 6:41
4
$\begingroup$

Since you want to compare the entire distributions, I'd recommend the two sample Kolmogorov-Smirnov test.

More information can be found here:

http://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ One of the assumptions is that the two samples are mutually independent. Given your description, it may be more appropriate to perform a one-sample KS test of the sample versus the parent population. $\endgroup$
    – soakley
    Commented May 13, 2014 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.