I am reading chapter 11 . Sampling Methods from the book : Pattern Recognition and Machine Learning by Bishop :
In the introduction , in short,he evaluates expectation of some function $f(z)$ with respect to a probability distribution $P(z)$ where $z$ is the random variable . He writes :
\begin{align}
E(f) = \int f(z)p(z)dz
\end{align}
Now he says that we suppose that such expectations are too complex to be evaluated exactly using analytical techniques . So we use sampling. And the idea behind sampling methods is to obtain a set of samples $z^{(l)}$, (where $l=1,....,L$) drawn independently from the distribution $P(z)$. This allows the expectation to be written as
\begin{align}
f^{\hat{}} = \frac{\sum_{l=1}^{L}f(z^{(l)})}{L}
\end{align}
Now further in the official solution for excercise 11.1 , he calculates :
\begin{align}
E[f^{\hat{}}] = \frac{1}{L}\sum_{l=1}^{L}\int f(z^{(l)}) p(z^{(l)})dz^{(l)}\tag 1
\end{align}
Now i did not understand this integral .
My argument , for simplicity , let us assume the underlying distribution $P(z)$ to be a one dimensional standard normal distribtion .
1.) When the author samples $z^{1}$, i assume that he drew a fixed $n$ values which would look something like : $z^{1}= \{0.2,0.30.2323,... ,0.8\}$ ,total $n$ values .
Now in that case how would the integeral in $(1)$ look like ? For example
what would be the limits of integration ?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If $P(z)$ is a one-dimensional normal distribution, then there is no $n$. Each sample $z^{(l)}$ is a single draw from $P(z)$, i.e. $z^{(l)}$ is just a real number. The integral is over the support of the distribution. In this case, the limits of integration are $(-\infty, \infty)$.
