Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

I have four sets of distribution $X_1$, $X_2$, $Y_1$ and $Y_2$. I would like to compare which pair is more overlapping: $X_1$ vs. $Y_1$, or $X_2$ vs. $Y_2$.

Is there any test or quantifier for me to deal with this issue?

share|improve this question
2  
What do you mean by overlapping? Same values in the 2 sets? Then you could simply count the common instances. Or are you referring to the degree of correlation between the vectors? Or the shape of the distribution? – Robert Kubrick Mar 29 '12 at 13:07
@RobertKubrick All the sets are being sampled, so overlapping means the population lies in the same sampling interval. – Rein Mar 30 '12 at 3:06
1  
Rein, in light of your response to Robert, what precisely do you mean by "more" overlapping? Two different senses come immediately to mind (and there are likely other useful definitions). One would be in terms of the joint probabilities of "overlapping" and another would be in terms of the measure of the range of overlap. A good way to clarify things is to make this question less abstract and tell us what practical problem you are having to deal with. (Please review our FAQ concerning such matters.) – whuber♦ Mar 30 '12 at 14:02
@Rein do you mean this overlap ? dmbru.net/Critical_Thinking/Normal_Intersection.html – Stéphane Laurent May 29 '12 at 5:04

closed as not a real question by whuber♦ May 29 '12 at 17:22

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, see the FAQ.

4 Answers

up vote 0 down vote

Possibly measuring relative entropy?

share|improve this answer
It is hard to use relative entropy, because there is always some cases set $X_i$ have zero probability but $Y_i$ is not zero or vice versa. – Rein Mar 30 '12 at 3:56
en.wikipedia.org/wiki/Rule_of_succession (No zeros :) ) – mepuzza Apr 28 '12 at 16:55
up vote 0 down vote

You can use the Kolmogorov-Smirnov test statistic as a measure of overlap, or more specifically one minus the statistic. If you have 2 continuous distributions and the distribution functions only have 1 point of intersection then one minus the KS stat is exactly the area of overlap between the 2 distributions. If there are more points of intersection then the stat does not give exact overlap area, but would still be a measure of the overlap.

share|improve this answer
up vote 0 down vote

Suppose you have $n$ observations from $X_1$ and $m$ from $Y_1$. Suppose the largest $Y$ is bigger than the largest $X$. Let $Y_1(1)$ be the minimum value of $Y_1$ and $X_1(n)$ be the maximum value of $X_1$. Then the interval from $Y_1(1)$ to $X_1(n)$ is a measure of the overlap. This will under estimate the overlap because the lower bound for $Y_1$ can be lower than $Y_1(1)$ and the upper bound on $X_1$ can be higher than $X_1(n)$. However as $n$ and $m$ get large (assuming $X_1$ and $Y_1$ have finite bounds the interval will approach the overlap. If the upper bound for $X_1$ happens to exceed the upper bound for $Y_1$ the overlap truncates at the upper bound for $Y_1$. The theory of extreme values may be helpful in determining how rapidly the minimum and maximum for each variable approaches its upper and lower bound.

share|improve this answer
Why do you say the the lower bound for $Y_1$ can be lower than $Y_1(1)$, when you just defined $Y_1(1)$ as the minimum of $Y_1$? I think you mean $Y_1$ is a sample from $Y$, right? So it's the lower bound of $Y$ that can be lower than $Y_1(1)$, no? Might be clearer to say that $X_1$/$Y_1$ are n/m sized samples from $X$/$Y$ at the start. – naught101 May 29 '12 at 3:23
I meant the samples from Y1 and X1. – Michael Chernick May 29 '12 at 4:00
up vote 0 down vote

If the $X_i$ and $Y_i$ are distributions then you're looking for a metricization on the space of distributions. Many such exist! Supnorm, $L_1$, $L_2$, $L_p$-norms, Bregman divergences, mutual information, &c.

Based on you asking for a test, however, I think that the $X_i$ and $Y_i$ are not distributions but instead some kind of random variables. These

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.