1
$\begingroup$

This is in context of Gaussian mixtures $$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$

Bishop mentions on Page-111

Also, the requirement that $p(\boldsymbol{x})\ge 0$, together with $\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})\ge$ $0$, implies $\pi_k\ge0$ for all $k$

How do we prove this?

A trivial counter-example that comes is ($k=2$), with a single Gaussian taken twice (i.e., $\boldsymbol{\mu_1}=\boldsymbol{\mu_2}$, $\boldsymbol{\Sigma_1}=\boldsymbol{\Sigma_2}$) and $\pi_1=1.2$, $\pi_2=-0.2$.

$\endgroup$
4
  • $\begingroup$ What do you want to prove? That if one of the elements in the sum is negative, the total can be negative? $\endgroup$
    – Tim
    Commented Oct 23, 2022 at 7:13
  • $\begingroup$ As mentioned, I want to prove the cited statement . And I don't know if the claim itself is correct -- the "counter-example" is for the same. $\endgroup$
    – muser
    Commented Oct 23, 2022 at 7:15
  • $\begingroup$ It's a requirement, what about it you want to prove? $\endgroup$
    – Tim
    Commented Oct 23, 2022 at 7:16
  • $\begingroup$ I want to prove the "implies $\pi_k \ge 0$" part. $\endgroup$
    – muser
    Commented Oct 23, 2022 at 7:17

1 Answer 1

0
$\begingroup$

You are misinterpreting the quote. The requirement follows from the fact that we need $p(\boldsymbol x)$ to be a proper probability distribution (non-negative, that integrates to 1), so we use a convex combination of the mixture components. Moreover, $\pi_k$ weights are interpreted as probabilities, which makes the model have an intuitive interpretation.

See also Intuition for why sum of gaussian RVs is different from gaussian mixture

$\endgroup$
6
  • $\begingroup$ I had earlier edited my question to correct the counter-example. $\endgroup$
    – muser
    Commented Oct 23, 2022 at 8:59
  • $\begingroup$ I agree to the probabilistic interpretation of $\pi_k$. As a requirement, it is fine (but the author didn't put it that way). $\endgroup$
    – muser
    Commented Oct 23, 2022 at 9:53
  • $\begingroup$ @abs regardless of the interpretation, the first part of the answer applies, the result needs to be proper distribution. Maybe the wording was unfortunate, but the author just lists a set of conditions that make it true. $\endgroup$
    – Tim
    Commented Oct 23, 2022 at 10:15
  • $\begingroup$ But is it true that a concave combination necessarily does not give a proper distribution? $\endgroup$
    – muser
    Commented Oct 23, 2022 at 10:18
  • $\begingroup$ As an aside, the author imposes $0\le\pi_k\le1$ on page-431 (9.8). This, with the cited statement in the question, seems confusing. $\endgroup$
    – muser
    Commented Oct 23, 2022 at 10:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.