Consider choosing $\theta^*$ that minimizes the expected absolute loss:

\begin{align} \tag{1} \int_{\Theta}|\theta-\theta^*|\pi(\theta|\mathbf{x})d\theta= \int_{-\infty}^{\theta^*}(\theta^*-\theta)\pi( \theta|\mathbf{x})d\theta+\int_{\theta^*}^{\infty}(\theta-\theta^*)\pi(\theta|\mathbf{x})d\theta \end{align}

Differentiating with respect to $\theta^*$ and equating to zero yields: \begin{align} \tag{2} \int_{-\infty}^{\widehat{\theta^*}}\pi(\theta|\mathbf{x})d\theta = \int_{\widehat{\theta^*}}^{\infty}\pi(\theta|\mathbf{x})d\theta \end{align}


I'm confused about the step taken from (1) to (2), which can be found on p. 12 (94) in online notes (Note: I replaced $\delta(\mathbf{x})$ by $\theta^*$) . A more detailed insight will be highly appreciated. My understanding is that it makes use of Leibniz rule. Here $\widehat{\theta^*}$ is the Bayes estimate (which turns out to be the median of $\pi(\theta|\mathbf{x})$), an argument that minimizes (1).

  • 1
    $\begingroup$ Could you explain what kind of mathematical object you mean by "$\delta$"? If it's not a number (the notation suggests it might be a function), could you further tell us what you mean by "differentiating" with respect to it would be? $\endgroup$
    – whuber
    Nov 7 '18 at 17:39
  • 2
    $\begingroup$ Please explain how one differentiates with respect to an estimator! The only way I can think of doing this in general is explained at stats.stackexchange.com/questions/369933, but it's unclear whether that's what you have in mind. $\endgroup$
    – whuber
    Nov 7 '18 at 17:50
  • 1
    $\begingroup$ Yes it's a well-known result. But it is derived using actual mathematics rather than fanciful operations like those you seem to be using! (That's not to denigrate such work--non-rigorous or even nonsensical forms of mathematics can provide some intuition. But that's not what you seem to be asking about.) Until we know more about what you mean by differentiating with respect to a function, none of your work will be in the least clear. $\endgroup$
    – whuber
    Nov 7 '18 at 18:19
  • $\begingroup$ @whuber thanks, you're right, I will reformulate the question. This is exactly why I found that proof confusing. $\endgroup$
    – AlexMe
    Nov 7 '18 at 18:21
  • 1
    $\begingroup$ @AlexMe You can avoid the Leibniz rule if you write $\int_{-\infty}^t(t-x)\pi(x)\,dx=t\int_{-\infty}^t \pi(x)\,dx -\int_{-\infty}^t x\,\pi(x)\,dx$. Then differentiate the first piece using the product rule, and the second piece using FTC. $\endgroup$
    – grand_chat
    Nov 7 '18 at 22:25