In Section 4.2 of the paper by Boucheron et al., the authors argue that the minimizer $f^*$ of the cost functional $$A(f) = \mathbb{E}\{\phi(-Yf(X))\}$$ is such that the classifier $g^*$, constructed from $f^*$ by $$ g^*(x) = \begin{cases} 1 & \text{if } f^*(x)\geq 0 \\ -1 & \text{otherwise} \end{cases}$$ is the Bayes classifier. Note that the cost function $\phi$ is a non-negative, strictly convex, and non-decreasing function.

In order to prove the claim, the authors argue that $\phi'$ is strictly increasing. Where is that coming from?

  • $\begingroup$ Quick reminder from Wikipedia, "A loss function is Fisher consistent if the population minimizer of the risk leads to the Bayes optimal decision rule." $\endgroup$ – Pardis May 22 '12 at 13:40
  • 4
    $\begingroup$ (This should be a comment to the reply but there are so many comments there it would be lost in the noise!) Note that $\phi$ is not assumed to have a second derivative. The conclusion follows from the assumptions that $\phi$ is nondecreasing and strictly convex. Proof: If $\phi'$ were not strictly increasing on a nontrivial interval $[a,b]$, then $\phi'$ would be constant on some nontrivial subinterval $[a_0,b_0]$, implying $\phi$ would not be strictly convex on $[a_0,b_0]$, contradicting the assumptions, QED. $\endgroup$ – whuber May 22 '12 at 14:53

Given that $\phi$ is twice differentiable, strictly convex and nondecreasing, you have that $\phi^{\prime\prime}(x)>0$ for all $x$ in the interior of its domain. This implies that $\phi^{\prime}(x)$ is strictly increasing (given that its derivative is positive).

  • $\begingroup$ Even when the second derivative of $f$ exists everywhere, the first sentence of this answer is not true. Take $f(x) = x^{2n}$ with $n \geq 2$. $\endgroup$ – cardinal May 22 '12 at 13:56
  • $\begingroup$ @cardinal that function is not "nondecreasing". $\endgroup$ – user10525 May 22 '12 at 13:57
  • $\begingroup$ That depends on its domain of definition. $\endgroup$ – cardinal May 22 '12 at 13:59
  • 1
    $\begingroup$ (+1) I hope you don't mind the typo correction and small tweak. :) $\endgroup$ – cardinal May 22 '12 at 14:39
  • 1
    $\begingroup$ Do you mean the derivative of $X^{2n}$ is $2nX^{2n-1}$? $\LaTeX$ code for that is $X^{2n}$ and $2nX^{2n-1}$ $\endgroup$ – Macro May 22 '12 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.