A classification for two mutually exclusive problem can be formulated by having a decision hinge on whether $P_0(x) > P_1(x)$ or $P_0(x) < P_1(x)$ where $P_0(x)$ and $P_1(x)$ are posterior probabilities. However, is is valid to simply sum all the posterior probabilities $P_0(x)$ and $P_1(x)$ a to obtain total counts for both groups?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ With what goal? What are you trying to use the sum for? $\endgroup$– DanicaCommented Jan 20, 2015 at 22:34
-
$\begingroup$ I'm using Bayes theorem to classify data into two groups, and I want to get a total count of data that falls either in group 1 or group 2. $\endgroup$– MarkCommented Jan 20, 2015 at 22:53
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Yes, if the events are exclusive, then you can sum their probabilities to obtain probability of observing any of the events.
If $E_1, E_2,\dots,E_k$ are mutually exclusive events, then by definition $P(\cup_{i=1}^k E_i) = \sum_{i=1}^k P(E_i)$. So if you have estimated probabilities for $E_1$ and $E_2$, then $\hat P(E_1 \cup E_2) = \hat P(E_1) + \hat P (E_2)$. Same with merging the levels of Dirichlet distribution.
If $\mathbf{X}$ follows multinomial distribution $\mathbf{X} \sim \mathcal{M}(n, \mathbf{p})$, then marginally $X_i$'s are binomial distributed $X_i \sim \mathcal{B}(n, p_i)$ and for $i\ne j$, the sum of $X_i$ and $X_j$ is also binomial $X_i + X_j \sim \mathcal{B}(n, p_i+p_j)$.
$\begingroup$
$\endgroup$
5
$\newcommand\E{\mathbb{E}}$It's valid in the following way:
Suppose that the label of each class is distributed according to $\mathrm{Bernoulli}(P_1(x))$. Then the expected number of instances of class one is the sum of the posterior probabilities.
This is a fairly reasonable model for class variables, though I wouldn't take the values it gives out too literally unless you have much better classification probability estimates than are typical.
-
$\begingroup$ Thank you for answering the question! I didn't go into detail about why I am summing the probabilities instead of using the decision rule I mentioned in the question. By summing the probabilities I get a more consistent rate of classification, whereas the decision rule method underestimates the instances in class one that happens to have a lot less instances than the other class two. I am not sure what the discrepancy between summing the posterior probabilities and using the decision rule say about the validity of either method. $\endgroup$– MarkCommented Jan 21, 2015 at 1:02
-
$\begingroup$ @Mark So what's your new decision rule exactly? $\endgroup$– DanicaCommented Jan 21, 2015 at 1:58
-
$\begingroup$ No new decision rule, I am talking about difference in total instances when I use the decision rule described in the question versus just summing the posterior probabilities to obtain total instances for a class. The summing method works better, but I wasn't sure if it was valid, or why it would be valid but I think it is since I am dealing with two mutually exclusive classes so the posterior probability is like Bernoulli distribution. $\endgroup$– MarkCommented Jan 21, 2015 at 2:08
-
$\begingroup$ Oh, you mean that you get a better estimate of the overall number of instances of class 1 by doing this sum than by summing the thresholded version? That makes some sense. $\endgroup$– DanicaCommented Jan 21, 2015 at 2:12
-
$\begingroup$ Yes that is exactly the case. Both methods preserve the total number of instances, the threshold version just underestimates the instances in class 1. Intuitively it makes sense that would be the case if I have a lot instances with class 1 posterior probability of less than 0.5, however, I don't know of any theoretical underpinning for such result. Again, first I wanted to make sure it was valid to sum posterior probabilities in the first place. $\endgroup$– MarkCommented Jan 21, 2015 at 4:46