3
$\begingroup$

I'm working on a naive Bayes classifier that calculates probabilities using a normal Gaussian distribution. This works very well when I am classifying something into two mutually exclusive buckets (e.g. spam vs. not-spam), but when I am working with a factor that is not easily classified that way (when the classifications are not mutually exclusive) I would like to express the result as a percentage.

When I combine the probability density of several factors (by multiplying them together) I tend to get a very small number and I would like to adjust that so I can express it in a 0-100 percent range, so it will be more easily understood. Is there another factor I can use to adjust the probability density into a percentage range?

For example: in the Wikipedia article for naive Bayes classifiers, there is an example of using height, weight and shoe size variables to classify a person as male or female. After computing the numbers, the posterior result for the female classification is 5.3778E-4 (or .00053778). Out of context that seems minuscule, but if you compare that number to the result for the male classification, the percentage would be 99.99% female. What factors (if any) could I apply to the posterior result to get that percentage, without the benefit of the male result to compare it to?

Improve this question
$\endgroup$
7
  • $\begingroup$ Why not work with logarithms of probabilities? $\endgroup$
    – charles.y.zheng
    Commented Mar 18, 2011 at 21:46
  • 1
    $\begingroup$ Technically, you could compute $p(data)$, which is the normalization factor. However, the easiest way to compute it is usually to compute the results for all classes and sum them up. Note that this works for any number of classes, not just two. $\endgroup$
    – SheldonCooper
    Commented Mar 18, 2011 at 21:46
  • 1
    $\begingroup$ @Matt Those numbers are not probabilities at all: they are probability densities. It makes no sense to force a bunch of them to sum to 1. (In fact, you can make them individually be greater than 1 merely by changing the units of measure for weight or whatever.) See the related question and analysis at stats.stackexchange.com/q/4220/919 . $\endgroup$
    – whuber ♦
    Commented Mar 19, 2011 at 4:48
  • $\begingroup$ @whuber that's right, the question is about probability densities. I'm wondering if I can apply a function or constant(s) (such as the normalization factor mentioned above by SheldonCooper) to transform the probability density "5.3778E-4" from the posterior value in the example to something like "99.99%", or is that not practical? $\endgroup$
    – Matt
    Commented Mar 19, 2011 at 7:10
  • 1
    $\begingroup$ @Matt It is practicable (it can be done) but its meaningfulness is doubtful. However, ratios of densities do make sense, as @ProbabilityIsLogic has suggested in a reply. For instance, you might re-express all the results as ratios relative to the largest one. They won't sum to unity--that is the part that is meaningless--but they will all lie between 0 and 1. $\endgroup$
    – whuber ♦
    Commented Mar 19, 2011 at 13:58

1 Answer 1

Reset to default
2
$\begingroup$

You can use any "monotonic" transformation of the probabilities as you choose (at least as far as I know). As long as your transformation preserves the ordering of probabilities, you will not be lead astray in your decision making. Personally, I prefer to use odds ratios. They seem to make intuitive sense to me, and I know what decision to make almost straight away (and the "region of uncertainty" is easily identified, perhaps odds between 0.1 to 10?).

my answer to this question goes through a worked example of how you would use the odds in naive Bayes classifier.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.