0
$\begingroup$

have a normal distribution. I would like to compare two input probabilities from this population to measure how "similar" they are. Everything is subjective but I wanted to be able to say that $x$ is more "similar" to $y$ than $z$ to $y$, using some sort of an equation against the normal distribution.

For example, if the mean of my population is 10, and my standard deviation is 3. I would like my simple algorithm to say find that two points (19 and 17) are more similar than two other points (9 and 10) simply because it's a lot less likely to get point 17 (since it's more than two sigmas away from the mean), thus getting another random point to be near that first point with the low probability, shows higher similarity than comparing two points that occur with much equally higher probabilities.

Using something like $P(X < p_1) - P(X < p_2)$ is not good enough because I may get $0$ if both points are the same. However, obtaining two points of $9$ and $9$, should score less similarity than two points ($20$ and $20$) since $20$ is a lot less likely to occur than $9$.

I feel like I need to use the difference above but somehow also use the mean and sigma to formulate a similarity "formula".

Is there an existing stat test that captures what I'm trying to do above? If not, does anyone have a suggestion as how I would go about solving the problem above?

$\endgroup$
  • 1
    $\begingroup$ It's not particularly clear from your discussion what you're really aiming to capture there. The difficulty is one could sit here making a hundred guesses about what you want; I think you need to be able to articulate its properties more completely. What underlying problem does this solve? $\endgroup$ – Glen_b Sep 5 '13 at 4:56
  • $\begingroup$ Assume I have a number (given the distribution in my example above). What's the probability that you would randomly generate that same number that I have? The trick is this is not just an exact match issue, but proximity problem as well. So, what's the likelihood that you randomly pick a number that's significantly close to the number I have. Provided both are part of the same normal distribution. $\endgroup$ – user2589308 Sep 5 '13 at 6:24
  • $\begingroup$ Lets say two numbers are pulled out of a hat, and the way it was done could have caused them to corrupt (change value). I would like to perform a "similarity" test to test how likely that those numbers were the same before corruption. If they match but fall right on the mean value then confidence is low because many random values would fall around the mean. But if they match or close to each other but far away from the mean, it would be more likely that they were the same but were corrupted since it's not very likely to get such values away from the mean. $\endgroup$ – user2589308 Sep 5 '13 at 6:27
  • $\begingroup$ I suspect I need to use one as a base, then use the mean and standard deviation of the population to calculate how similar or dissimilar the other number is. Does this make sense? $\endgroup$ – user2589308 Sep 5 '13 at 6:27
  • $\begingroup$ "What's the probability that you would randomly generate that same number that I have?" ... well, zero. But you might write it as $f(x)dx$, in which case $(20|20)$ is relatively less likely than $(9|9)$. Your notions are incompatible with each other. "significantly close" doesn't mean anything to me. "how likely that those numbers were the same before corruption"; again, zero, but if you extend it to some kind of proximity, the question becomes one about the corruption process, which you haven't specified. "I suspect I need to use one as a base" -- why? is one inherently a base? $\endgroup$ – Glen_b Sep 5 '13 at 6:42
1
$\begingroup$

From what I understand, you are trying to combine in a single metric two things: a) the distance between two possible values a random variable can take and b) the probability that the random variable will take values "around there". It does seem that the way to do it, is to measure the probability mass that is enclosed by these two points, and then declare as more "similar" (in your combined sense) the pair that encloses the smaller probability mass. You discuss this yourself, but you write that it doesn't cover the case where the value is exactly the same. But this can be overcome by simply writing this metric as a piecewise expression, say

$$M = P(X<p_1) - P(X<p_2),\; p_1\neq p_2$$ $$M = P(X<p_1), \; p_1=p_2$$

or compactly using the indicator function

$$ M = P(X<p_1) - P(X<p_2)I_{\{p_1\neq p_2\}},\;\; p_1\ge p_2$$

After all, if the two points have zero distance between them, it is only "fair" to take into account only the probability that must be accumulated in order for the random variable to "reach there". The higher $M$ for a pair of points compared to another pair of points, the less "similar" is the first pair, in your sense.

ADDENDUM : Checking the conceptual consistency of the measure Since by the OP's comments to my answer it seems that the OP believes that the answer may be useful, I note that some simple "conceptual consistency" checks must be run, by trying various combinations of pairs of points and see whether this approach gives a logical result. I give one example:

The way it is defined above, it means that the pair of points $(x_1,x_1)$ will always be considered "less similar" than any pair of $(x_1, x_i)$ , $x_1 > x_i$. Is this reasonable regarding the purpose at hand?

$\endgroup$
  • $\begingroup$ That's a great idea! So simple yet I don't know how it went past me. Thank you! $\endgroup$ – user2589308 Sep 7 '13 at 5:24
  • $\begingroup$ Actually. Now that I got enough sleep and just starting to think about it. This will mean that two numbers differing slightly around the mean will be more similar than two that are exactly equal but equal to the mean. It seems like I shouldn't take P(X<x) if the two values are the same. It seems like I should have some sort of a penalty factor that diminishes as you go away from the mean but shouldn't punish the value so severely that would make two numbers around the mean more similar than two exactly the same around the mean. So you are right in your edits above. $\endgroup$ – user2589308 Sep 7 '13 at 18:13
  • $\begingroup$ doesn't this metric increase when you move p1 and it touches p2 ? It goes from near 0 (p1 near p2) to P(x<p_1) ... $\endgroup$ – josinalvo Dec 4 '13 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.