1
$\begingroup$

I am trying to understand some notation used in papers about Bayesian variational inference. In some papers that use mean-field approximation to fit a probabilistic model, they describe coordinate ascent algorithms which have updates with the symbol $\propto$ for the closed-form update of some parameters.

From what I've seen, it seems these $\propto$ always tend to be $\propto \: exp( . )$

For example these: enter image description here (Gopalan, Prem K., Laurent Charlin, and David Blei. "Content-based recommendations with poisson factorization." Advances in Neural Information Processing Systems. 2014., page 5)

enter image description here (Cemgil, Ali Taylan. "Bayesian inference for nonnegative matrix factorisation models." Computational intelligence and neuroscience 2009 (2009)., page 5)

And I'm really wondering what would that $\propto$ translate to in an algorithmic procedure.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ $\propto$ means "proportional to", usually used when a constant factor term has been dropped from one or both sides. $\endgroup$
    – shimao
    Commented May 3, 2018 at 21:50
  • $\begingroup$ Thanks, but I meant to ask, how does that work in terms of an algorithm? Let's say you were to design a program that follows an iterative update a := exp(b) and then something else - that means you assign the right hand side to a at each iteration. But since $\propto$ is proportional and not equal, how would that work in an algorithm or computer software? $\endgroup$
    – anymous.asker
    Commented May 4, 2018 at 7:34

1 Answer 1

Reset to default
1
$\begingroup$

The proportion is intended to hide the terms that are constant in the expression and highlight the rest. For variational Bayes updates, this hidden proportionality constant usually contains a normalization term (i.e., a term that makes the expression sum to unity), but it may also contain other terms depending on the context.

In derivations, it is safe to hide these terms because they do not impact the location of the optima, only the value of the function at the optima. In software, though, VB updates are usually coupled, so you often need to compute and apply the constants.

As a side note, some authors prefer to write log-update equations e.g., $$ \log q(Z) = \mathbb{E}[\log p(Z,W)] + \text{const}, $$ where $\text{const}$ is an additive constant that can be interpreted the same way.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for answering! That's also my first thoughy when I see $\propto$, but it turns out it means something different here: calculate the expression in the right, then divide it by the sum of all the elements (calculated like that) in that row so that rows sum to 1. $\endgroup$
    – anymous.asker
    Commented Sep 15, 2018 at 21:52
  • $\begingroup$ Yes, that is the normalization factor I am talking about. $\endgroup$
    – scherm
    Commented Sep 15, 2018 at 22:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.