3
$\begingroup$

I'm working on a naive Bayes classifier that calculates probabilities using a normal Gaussian distribution. This works very well when I am classifying something into two mutually exclusive buckets (e.g. spam vs. not-spam), but when I am working with a factor that is not easily classified that way (when the classifications are not mutually exclusive) I would like to express the result as a percentage.

When I combine the probability density of several factors (by multiplying them together) I tend to get a very small number and I would like to adjust that so I can express it in a 0-100 percent range, so it will be more easily understood. Is there another factor I can use to adjust the probability density into a percentage range?

For example: in the Wikipedia article for naive Bayes classifiers, there is an example of using height, weight and shoe size variables to classify a person as male or female. After computing the numbers, the posterior result for the female classification is 5.3778E-4 (or .00053778). Out of context that seems minuscule, but if you compare that number to the result for the male classification, the percentage would be 99.99% female. What factors (if any) could I apply to the posterior result to get that percentage, without the benefit of the male result to compare it to?

$\endgroup$
7
  • $\begingroup$ Why not work with logarithms of probabilities? $\endgroup$ Mar 18 '11 at 21:46
  • 1
    $\begingroup$ Technically, you could compute $p(data)$, which is the normalization factor. However, the easiest way to compute it is usually to compute the results for all classes and sum them up. Note that this works for any number of classes, not just two. $\endgroup$ Mar 18 '11 at 21:46
  • 1
    $\begingroup$ @Matt Those numbers are not probabilities at all: they are probability densities. It makes no sense to force a bunch of them to sum to 1. (In fact, you can make them individually be greater than 1 merely by changing the units of measure for weight or whatever.) See the related question and analysis at stats.stackexchange.com/q/4220/919 . $\endgroup$
    – whuber
    Mar 19 '11 at 4:48
  • $\begingroup$ @whuber that's right, the question is about probability densities. I'm wondering if I can apply a function or constant(s) (such as the normalization factor mentioned above by SheldonCooper) to transform the probability density "5.3778E-4" from the posterior value in the example to something like "99.99%", or is that not practical? $\endgroup$
    – Matt
    Mar 19 '11 at 7:10
  • 1
    $\begingroup$ @Matt It is practicable (it can be done) but its meaningfulness is doubtful. However, ratios of densities do make sense, as @ProbabilityIsLogic has suggested in a reply. For instance, you might re-express all the results as ratios relative to the largest one. They won't sum to unity--that is the part that is meaningless--but they will all lie between 0 and 1. $\endgroup$
    – whuber
    Mar 19 '11 at 13:58
2
$\begingroup$

You can use any "monotonic" transformation of the probabilities as you choose (at least as far as I know). As long as your transformation preserves the ordering of probabilities, you will not be lead astray in your decision making. Personally, I prefer to use odds ratios. They seem to make intuitive sense to me, and I know what decision to make almost straight away (and the "region of uncertainty" is easily identified, perhaps odds between 0.1 to 10?).

my answer to this question goes through a worked example of how you would use the odds in naive Bayes classifier.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.