Suppose that
- $h \in \mathcal{H}$ is a hypothesis in some class of binary classifiers $\mathcal{H}$,
- $\mathcal{D}_n$ is a training dataset of size $n$,
- $\mathcal{L}$ is the loss function for the binary classification problem defined as $$ \mathcal{L}(x,h) = \begin{cases} 1, & s(x) \not= h(x) \\ 0, & \text{otherwise} \end{cases} $$ where $s(x)$ is the system we are trying to model,
- $R_e(h)$ is the empirical risk of $h$ over a given dataset $\mathcal{D}_n$ defined as $$ R_e(h) = \frac1n\sum_{i=1}^{n}\mathcal{L}(x_i, h(x_i)) $$
- and $R(h)$ is the true risk of the hypothesis $h$.
How can I show that $$ \mathbb{E}_{\mathcal{D}_n}\left[R_e(h)\right] = R(h) $$ where the expectation on the LHS is over all possible training datasets $\mathcal{D}_n$ of size $n$.
What I've tried so far
Since $$ R_e(h) = \frac1n\sum_{i=1}^{n}\mathcal{L}(X_i, h(x_i)) $$ then \begin{align} \mathbb{E}_{\mathcal{D}_n}\left[R_e(h)\right] &= \int_{\mathcal{D}_n}{R_e(h)p(\mathcal{D}_n)} \\ &= \frac{1}{n}\int_{\mathcal{D}_n}{\sum_{x_i \in \mathcal{D}_n}\mathcal{L}(x_i, h)p(\mathcal{D}_n)} \end{align} I now want to manipulate this to convert it to $$ R(h) = \int_{x}{\mathcal{L}(x,h)p(x)dx} $$ I thought of grouping all the $x_i$'s out of the above equation, but I couldn't find a way to get the $p(x)$ term into the picture and this is where I am stuck. I am looking for progressive hints that will help me solve this myself.