In case of VAE, it is said that the posterior distribution is intractable because the marginal likelihood is intractable.

enter image description here

My understanding as to why marginal likelihood is intractable:

z can have infinitely many possible values, which means the integration will have infinite standard operations. Hence, it will not be a closed form solution and thus is intractable.

Is my understanding correct? If not, could anyone please correct me on this?

Another question: can latent variable z be discrete? Since integration is only applied for continuous variables, if z is discrete, I think p(x) will be as follows:

p(x) = ∑𝑧 𝑝(𝑥|𝑧)𝑝(𝑧)

Here, z will have fixed number of discrete values and p(x) will hence be tractable. With this, I am basically trying to understand whether or not the latent variables can be discrete in nature and marginal likelihood be tractable due to it in case of Variational Inference.

One more confusion that I had is: since z is latent (hidden), how can the values of z be known to be used in integration / summation?

  • 2
    $\begingroup$ What do you mean by infinitely many possible values? Integrals computed are over continuous sets. This does not mean the integral cannot be computed in all cases. $\endgroup$
    – Xi'an
    Commented May 12, 2022 at 13:05
  • $\begingroup$ Are integrals computable even when we want to consider ALL values of continuous variable? Won't it be incomputable because there will be infinitely many possible values of continuous variable? Example: 1.3, 1.32, 1.322, 1.3221, 1.3222, 1.32444 and the list continues. @Xi'an $\endgroup$
    – Curious
    Commented May 12, 2022 at 13:14
  • $\begingroup$ Also, besides intractability, the post consists of two other questions. Could you please provide me insight on those too? Or open the questions? @Xi'an $\endgroup$
    – Curious
    Commented May 12, 2022 at 13:16
  • 1
    $\begingroup$ @Curious integrals only work for continuous variables. For discrete variables, you need to pretend that they are continuing to be able to use them. I recommend you start by checking the linked thread and refreshing your knowledge of calculus, at it might be needed to understand it better khanacademy.org/math/integral-calculus $\endgroup$
    – Tim
    Commented May 12, 2022 at 13:23
  • 1
    $\begingroup$ It seems like you are completely missing the concept of an integral. For instance,$$\int_0^1 e^{-z}\,\text dz=1-e^{-1}$$is tractable. $\endgroup$
    – Xi'an
    Commented May 12, 2022 at 13:28


Browse other questions tagged or ask your own question.