Predictive Distribution is an expectation? The predictive distribution is defined as follows
$ p(\hat{y} | X) = \int p(\hat{y} | X) p(\theta | X) d\theta$
where $\hat{y}$ is unobserved value, $\theta$ represents parameters and $X$ denotes observed values (i.e data set.)
Two questions:

*

*Does the integral need to be computed for every new observation? or should the posterior be interpreted as distribution over all posible new observations?


*Is the integral equivalent to an expection? i.e:
$ p(\hat{y} | X) = E_{\theta | x} [p(\hat{y} | X)]$
If so,  how should  this  expectation  be interpreted ? is it a point estimate for $\hat{y}$ or how does it somehow represent a distribution?
 A: The definition of the posterior predictive distribution is,
$$
p(\hat y \mid X) = \int p(\hat y \mid \theta) p(\theta \mid X) d\theta  \quad (*)
$$
where $p(\hat y \mid \theta)$ is the likelihood of your model and $p(\theta \mid X)$ is the posterior distribution of $\theta$ after observing $X$.
The quantity $p(\hat y \mid X)$ can indeed be seen as an expectation.
Your model is $p(\hat y \mid \theta)$ and after observing $X$, you knowledge on the model parameter $\theta$ is represented by the posterior distribution $p(\theta \mid X)$.
From the integral above we see that $p(\hat y \mid X)$ is the expectation of $p(\hat y \mid \theta)$ given that the distribution of $\theta$ is $p(\theta \mid X)$, i.e:
$$
p(\hat y \mid X) = \mathbb E_{\theta \mid X} \left [ p(\hat y \mid \theta )\right ].
$$
Another way to see $p(\hat y \mid X)$ is think about it as the sum across $\theta$ of $p(\hat y \mid \theta )$, i.e the probability of $\hat y$ given the model is $\theta$, times your current knowledge of the probability of this model $p(\theta \mid X)$.
So it completely represents a distribution (a distribution marginalized over $\theta$).
If you wanted a point estimate of $\hat y$ you could take for example the expectation of this distribution:
$$
\int \hat y p(\hat y \mid X)d\hat y.
$$
Last, if I understand your first question, in the general case if you want to evaluate $p(\hat y \mid X)$ for different values of $\hat y $ you will need to recompute the integral in $(*)$ each time.
