Let $X$ be a random variable with values in $\mathcal{X}$ and $Y$ a random variable with values in $\{0,1\}$. Is it true that : $$\mathbb{P}(g(X) \ne Y) = \int_\mathcal{X} \mathbb{P}\big(g(X)\ne Y \hspace{1mm} \mid X=x\big)d\mathbb{P}_X(x)$$ ?
I agree with this equality in the case where $X$ is discrete (i.e $\mathcal{X}$ is a countable set), but I can't prove why it holds in the general setting where $X$ is not discrete and does not have a density function. Actually, I can't prove the equality when $X$ has a density neither.
I came across this equality in at least 2 articles about statistical learning theory :
- first article, page 4, proof of theorem 1.1
- second article, page 16, proof of proposition 1
The statements, whose proof uses the equality above, say that the Bayes classifier realizes the minimum of the risk $\mathbb{P}(g(X)\ne Y)$ over all measurable functions $g$.