# Convex cost functions and Fisher consistency

In Section 4.2 of the paper by Boucheron et al., the authors argue that the minimizer $f^*$ of the cost functional $$A(f) = \mathbb{E}\{\phi(-Yf(X))\}$$ is such that the classifier $g^*$, constructed from $f^*$ by $$g^*(x) = \begin{cases} 1 & \text{if } f^*(x)\geq 0 \\ -1 & \text{otherwise} \end{cases}$$ is the Bayes classifier. Note that the cost function $\phi$ is a non-negative, strictly convex, and non-decreasing function.

In order to prove the claim, the authors argue that $\phi'$ is strictly increasing. Where is that coming from?

• Quick reminder from Wikipedia, "A loss function is Fisher consistent if the population minimizer of the risk leads to the Bayes optimal decision rule." – Pardis May 22 '12 at 13:40
• (This should be a comment to the reply but there are so many comments there it would be lost in the noise!) Note that $\phi$ is not assumed to have a second derivative. The conclusion follows from the assumptions that $\phi$ is nondecreasing and strictly convex. Proof: If $\phi'$ were not strictly increasing on a nontrivial interval $[a,b]$, then $\phi'$ would be constant on some nontrivial subinterval $[a_0,b_0]$, implying $\phi$ would not be strictly convex on $[a_0,b_0]$, contradicting the assumptions, QED. – whuber May 22 '12 at 14:53

Given that $\phi$ is twice differentiable, strictly convex and nondecreasing, you have that $\phi^{\prime\prime}(x)>0$ for all $x$ in the interior of its domain. This implies that $\phi^{\prime}(x)$ is strictly increasing (given that its derivative is positive).
• Even when the second derivative of $f$ exists everywhere, the first sentence of this answer is not true. Take $f(x) = x^{2n}$ with $n \geq 2$. – cardinal May 22 '12 at 13:56
• Do you mean the derivative of $X^{2n}$ is $2nX^{2n-1}$? $\LaTeX$ code for that is $X^{2n}$ and $2nX^{2n-1}$ – Macro May 22 '12 at 14:57