Skip to main content
Cross Validated

Return to Question

added 181 characters in body
Source Link
Pierre Gloaguen
Pierre Gloaguen
  • 1
  • 1

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician, or this discussion for instance) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int \exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed in those references. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int \exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed in those references. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician, or this discussion for instance) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int \exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed in those references. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?

Typos in the math equations
Source Link
Pierre Gloaguen
Pierre Gloaguen
  • 1
  • 1

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\lbrace, $$$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\lbrace\text{d}z_k < \infty. $$$$ \int \exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed in those references. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\lbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\lbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int \exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\rbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed in those references. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?

Source Link
Pierre Gloaguen
Pierre Gloaguen
  • 1
  • 1

Conditions of applications for coordinate ascent variational inference?

In every reference about coordinate ascent variational inference for the mean field family (Chapter 10 Of the book of C.Bishop Pattern recognition and machine learning, or the review article of Blei et al 2017, Variational inference, a review for statistician) we find the main result which is:

$$ q^*(z_k) \propto\exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\lbrace, $$ which, as $q^*(z_k)$ is a probability distribution function, only as a sense when $$ \int exp\left\lbrace \mathbb{E}_{-k} \left[ \log p(z_k|z_{-k},x) \right] \right\lbrace\text{d}z_k < \infty. $$ However, to the best of my knowledge, this hypothesis is never discussed. Does it always hold in the case of mean field variational inference, and if so, why? If not, when does it hold?