I am studying logistic regression from the book Advanced Data Analysis from an Elementary Point of View which states the following on page 280:
“We minimize the mis-classification rate by predicting whichever class is more likely”:
Let $\hat Y(x)$ be our predicted class, either $0$ or $1$. Our error rate is then $P(Y \neq \hat Y)$ . Show that $P(Y \neq \hat Y) = E[(Y −\hat Y)^2]$ .
Further show that $E[(Y− \hat Y)^2|X=x] = Pr(Y=1|X=x)(1−2\hat Y(x))+\hat Y^2(x)$.
Conclude by showing that if $Pr(Y = 1|X = x ) > 0.5$, the risk of mis-classification is mini- mized by taking $\hat Y = 1$, that if $Pr(Y = 1|X = x) < 0.5$ the risk is minimized by taking $\hat Y = 0$, and that when $Pr(Y = 1|X = x) = 0.5$ both predictions are equally risky.
I am a bit lost by this approach, as I have seen the derivations for the Bayes classifier using $L_2$ loss in the continuous case, and using 0-1 loss in the discrete case in ESL (page 18), but nothing of this sort before.
How should one approach this problem ?