I have two set of n-dimensional multivariate data, with the assumption that both set of data is normally distributed. When I get a new data point, my goal is to classify it into one of the two sets. The sum of probability of my new data point belonging to both the distribution therefore should be 1. I am using this to calculate the probability of the new data point belonging to each of the distribution. However, I am getting probability scores in the range 0 to 1 for both the cases resulting in the sum of probability being sometimes greater than 1 and sometimes less than 1. I don't know much statistics, is there any way to combine probability in the range 0 to 1, and explain it mathematically, or is there some other method to get classify new data into one of the two distributions, which satisfies the probability law of sum being equal to 1? Thanks in advance.
1 Answer
Your link is not calculating the probability that an observation came form a particular distribution, but instead the probability that if another sample observation were drawn from that distribution you would see the original observation or something as extreme or more extreme than the original observation.
If your original observation is not extreme in either of the two distributions then it is possible to get the situation you describe, with the two calculations adding up to more than $1$. If your original observation is extreme in both distributions you can get the two calculations adding up to less than $1$. There is not reason to expect the two calculations to add to $1$.
One approach could be to compare the likelihoods of your observation under the two distributions (for continuous distributions proportional to the densities at that point, while your earlier calculations were related to the cumulative distribution functions) and then have probabilities calculated as $$\frac{f_1(x)}{f_1(x)+f_2(x)} \text{ and }\frac{f_2(x)}{f_1(x)+f_2(x)}$$ which will then clearly meet the conditions you have set. You would get the same result using Bayes' theorem if you had prior probabilities for the two original distributions of $\frac12$ each.
-
$\begingroup$ Thanks for correcting me. How do I calculate the likelihood of my sample belonging to a distribution as suggested, is there any way to do it in python? Sorry, I don't know much stats. $\endgroup$ Commented May 21, 2021 at 9:50
-
$\begingroup$ It would be difficult: I do not know your original distributions, and even if I did then I would use R rather than python $\endgroup$– HenryCommented May 21, 2021 at 9:53
-
$\begingroup$ Thanks for the help! Would look if there's some way in R to do that. $\endgroup$ Commented May 21, 2021 at 10:29
-
$\begingroup$ The density for a multivatiate normal is $(2\pi)^{-\frac{k}{2}}\det(\boldsymbol\Sigma)^{-\frac{1}{2}} \, e^{ -\frac{1}{2}(\mathbf{x} - \boldsymbol\mu)^{{{\!\mathsf{T}}}} \boldsymbol\Sigma^{-1}(\mathbf{x} - \boldsymbol\mu) }$ $\endgroup$– HenryCommented May 21, 2021 at 10:59