3
$\begingroup$

I am a bit confused about the usage of probability density functions in machine learning. Lets say I have two classes $C_1$ and $C_2$ which are lets say gaussian . Then I can classify a point $x$ by taking the class for which the posterior is bigger. So I just calculate the following two expressions: $$P(x|C_1)P(C_1) \;and\; P(x|C_2)P(C_2)$$

When doing such calculations in our lecture we always talked about probabilities. As far as I know we always "acted" as if these are probabilities but this isn't technically right? I mean plugging in $x$ into the functions doesn't give you a "probability" but I am assuming that it gives you something proportional to the probability.

$\endgroup$
2
  • $\begingroup$ Your $P(x\mid C_1)$ is presumably a conditional probability or density for $x$ given $C_1$ rather than a probability (more likely to be a probability if $x$ only takes discrete values, more likely to be a density if $x$ is a continuous random variable). You then use this as being proportional to the likelihood for $C_1$ having observed $x$: $\mathcal L(C_1 \mid x)$ (neither a probability nor a density as it does not have to sum or integrate to $1$), which you then multiply by the prior density or probability for $C_1$ to give something proportional to posterior for $C_1$ having observed $x$ $\endgroup$
    – Henry
    Sep 4, 2022 at 16:49
  • $\begingroup$ It would be good to make your definition of $P$ explicit; following fairly conventional definitions a statistician might assume you actually mean probability rather than say a marginal or conditional density (for which something like say $f$ would be more common) $\endgroup$
    – Glen_b
    Sep 4, 2022 at 22:46

1 Answer 1

1
$\begingroup$

The formula gives you joint probabilities

$$ P(x, C_i) = P(x|C_i) \, P(C_i) $$

but you are presumably using Bayes theorem here, so this should be

$$ P(C_i|x) = \frac{P(x|C_i) \, P(C_i)}{\sum_i \, P(x|C_i) \, P(C_i) } $$

Where you don't need the denominator if you want to do things like optimization, sampling, or comparing the probabilities. This lets us ignore the denominator. If you were interested in the conditional probabilities themselves, you'd need the denominator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.