# Probabilistic model: what's the probability of this model?

The above model is from Berkeley cs294-112 Page 23. It is said that $$p(O_{1:T},s_{1:T}, a_{1:T}) =p(s_1)\prod_{t}p(s_{t+1}|s_t,a_t)p(O_t|s_t,a_t)\tag 1$$

I'm quite confused about this solution: where is $$p(a_t|s_t)$$?

I'm thinking whether this has something to do with the assumption that $$p(a_t|s_t)$$ is a uniform distribution, but it uses $$=$$ instead of $$\propto$$ and the result is further used in the computation of the Evidence Lower Bound. How is Eq.(1) derived?

• can you point us to the page you saw this formula? – gunes Feb 7 '19 at 8:58
• @gunes It is in Page 23 – Maybe Feb 7 '19 at 13:02

It seems they assume (for simplicity) that $$p(a_t|s_t)$$ is uniform (they explicitly say so in page 9 (Backward messages)), as you said, and therefore its density function is $$f(a_t|s_t) = 1$$ (they assumed standard uniform), and since it is 1 it disappears from the formula.
• I know that $p(a_t|s_t)$ is uniform, but what is $f(a_t|s_t)$? – Maybe Feb 7 '19 at 11:09
• If $a_t$ is a continuous random variable, then $p(a_t)$ is always 0. It is common in papers to write $p(x)$ as a general notation, that refers to the probability if it is discrete and to density ($f(x)$) if it is continuous. In this case since it is continuous, by $p$ they actually mean $f$ (density function). – Orielno Feb 7 '19 at 18:53
• Hi, @Orielno. Thanks for your clarification. But I still do not understand why you said that the density function of a uniform distribution was 1? What do you think $p(O_{1:T}, s_{1:T}, a_{1:T})$ would be if we did not ignore anything? – Maybe Feb 13 '19 at 8:49
• Then it would be as you said, we would add $p(a_t | s_t)$ – Orielno Feb 14 '19 at 11:03
The probability model is $$p(O_{1:T},s_{1:T},a)=p(s_1)\prod_tp(s_{t+1}|s_t,a_t)p(O_t|s_t,a_t)p(a_t|s_t)$$ taking logarithm, we have $$\log p(O_{1:T},s_{1:T},a)=\log p(s_1)+\sum_t\log p(s_{t+1}|s_t,a_t)+\log p(O_t|s_t,a_t)+\log p(a_t|s_t)$$ because $$p(a_t|s_t)$$ is uniform, the last term is a constant term. This means it does not contribute to the discussion about the ELBO, so the instructor omitted it for simplicity.