Importance sampling - computing the mean of unnormalised importance weights

Question

I am completing an assignment for self-study, and am experiencing some confusion over some elementary algebra concerning importance sampling.

The context is as follows:

Given a random distribution $p(x)$ on $x = [x_1, ..., x_D]^T \in \mathbb{R}^D$, suppose we want to perform inference $\mathbb{E}_{p(x)}[f(x)]$ using importance sampling, with $q(x)$ as the proposal distribution. According to importance sampling we draw $L$ IID samples $x^{(1)}, x^{(2)}, ..., x^{(L)}$ from $q(x)$ and we have $$\mathbb{E}_{q(x)}[u_i] \approx \frac{1}{\sum^L_{i=1}u_i} \sum^L_{i=1} f(x^{(i)})u_i$$ where $u_i$ are the (unnormalised) importance weights $u_i = \frac{p(x^{(i)})}{q(x^{(i)})}$.

Computing the mean of the unnormalised importance weights.

Now if I want to compute the mean of the unnormalised importance weights under the proposal distribution $\mathbb{E}_{q(x)}[u_i]$, and assuming that $p$ is already normalised, is the following argument valid?

$$\begin{align} \mathbb{E}_{q(x)}[u_i] &= \mathbb{E}_{q(x)}\left[ \frac{p(x^{(i)})}{q(x^{(i)})} \right] \\ &= \int \frac{p(x^{(i)})}{q(x^{(i)})} q(x) dx\\ &= \int \frac{p(x^{(i)})}{q(x^{(i)})} q(x^{(i)}) dx^{(i)} \\ &= \int p(x^{(i)}) dx^{(i)} \\ &= 1 \end{align}$$

I have a feeling there is something wrong with this argument, in particular in going from the 2nd to the 3rd equality. The confusion is whether it is appropriate to "cancel" the proposal distributions as I have done. Whilst grappling with this, I tried to write down exactly what each quantity is.

In words, I understand that I the unnormalised importance weights $u_i$ are a deterministic, scalar function of a random vector $x^{(i)} \in \mathbb{R}^D$, and hence are random variables. And that in computing the expectation, I am drawing $x^{(i)} \sim q(x)$, computing the ratio $u_i = p(x^{(i)}) / q(x^{(i)})$, and taking an average of this by weighting this with all possible values that $x$ can take under the proposal distribution $q(x)$. However I am still unsure.

Some assistance would be greatly appreciated.

Answer 1

The confusion stands with the simultaneous use of $x$ and $x^{(i)}$: the $x^{(i)}$'s are iid from $q(\cdot)$ and hence what need be checked is that \begin{align} \mathbb{E}_{q}[u(x)] &= \mathbb{E}_{q}\left[ \frac{p(x)}{q(x)} \right] \\ &= \int_{\mathbb R^D} \frac{p(x)}{q(x)} q(x) \text dx\\ &= \int_{\mathbb R^D} p(x) \text dx \\ &= 1 \end{align}