9
$\begingroup$
  1. Consider the quadratic loss $L(\theta,\delta)=(\theta-\delta)^2$, with prior given $\pi(\theta)$ where $\pi(\theta)\sim U(0,1/2)$. Let $f(x|\theta)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}(x), \theta>0$ the likelihood. Find the Bayes estimator $\delta^\pi$.

  2. Consider the weighted quadratic loss $L_w(\theta,\delta)=w(\theta)(\theta-\delta)^2$ where $w(\theta)=\mathbb{I}_{(-\infty,1/2)}$ with prior $\pi_1(\theta)=\mathbb{I}_{[0,1]}(\theta)$. Let $f(x|\theta)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}(x), \theta>0$ be the likelihood. Find the Bayes estimator $\delta^\pi_1$.

  3. Compare $\delta^\pi$ and $\delta^\pi_1$

First I noticed that $f(x|\theta)\sim Beta(\theta,1)$, and I assumed that that is the likelihood, otherwise I don't get any posterior, then $$\pi(\theta|x)\propto f(x|\theta)\pi(\theta)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}*2\mathbb{I}_{(0,1/2)}(\theta)\sim Beta(\theta,1)$$ so the Bayes estimator with respect to quadratic loss is $$\mathbb{E}[\pi(\theta|x)]=\frac{\theta}{\theta+1}$$

I'm looking in the book The Bayesian Choice and there is a theorem about the Bayes estimator associated with weighted quadratic loss and it is given by $$\delta^\pi(x)=\frac{\mathbb{E}^\pi[w(\theta)\theta|x]}{\mathbb{E}^\pi[w(\theta)|x]}$$

Can someone explain to me how I calculate it?

What I tried is:

$$\delta^\pi(x)=\frac{\frac{\int \theta w(\theta)f(x|\theta)\pi(\theta)d\theta}{\int w(\theta)f(x|\theta)\pi(\theta)d\theta}}{\frac{\int f(x|\theta)\pi(\theta)d\theta}{\int w(\theta)f(x\theta)\pi(\theta)d\theta}}$$

I know that the support is $[0,\frac{1}{2}]$, but when I tried to integrate in the numerator

$$\int \theta w(\theta)f(x|\theta)\pi(\theta)d\theta=\int_0^\frac{1}{2}\theta\theta x^{\theta-1}d\theta=\frac{1}{x}\int_0^\frac{1}{2}\theta^2 x^\theta d\theta$$

I get no good results.

$\endgroup$
  • 1
    $\begingroup$ Isn't $w(\theta)$ nonnegative here? $\endgroup$ – Juho Kokkala Mar 16 '16 at 15:19
  • 3
    $\begingroup$ I don't understand your remark about "just for $w(\theta)$ nonnegative," because (1) a loss function won't ever become negative and (2) your loss function cannot be negative anyway. $\endgroup$ – whuber Mar 16 '16 at 15:19
  • $\begingroup$ @whuber Gosh, now I realized my idiocy, I was looking at the indicator support $\endgroup$ – user72621 Mar 16 '16 at 15:58
7
+50
$\begingroup$

First, note that I corrected the original wording of the question wrt the indicator functions in your likelihood definitions as they have to be functions of $x$ not $\theta$. Hence the likelihood is $$f(x)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}(x)$$ that clearly integrates to one: $$\int_0^1 \theta x^{\theta-1}\text{d}x = 1$$

Second, the posterior in $\theta$ is not a Beta function since as indicated by Greenparker $$\pi(\theta|x)\propto\, \mathbb{I}_{[0,1/2]}(\theta)\theta x^{\theta-1}\propto \mathbb{I}_{[0,1/2]}(\theta)\,\theta \exp\{\log(x)\theta\}$$ Due to the constraint on the values of $\theta$ it is not a Gamma distribution either, but a truncation of the Gamma distribution.

Hence the Bayes estimator is the posterior expectation $$\begin{align*}\mathbb{E}[\theta|x] &= \int_0^{1/2} \theta\times\theta \exp\{\log(x)\theta\}\text{d}\theta \Big/ \int_0^{1/2} \theta \exp\{\log(x)\theta\}\text{d}\theta\\&= \int_0^{1/2} \theta^2 \exp\{\log(x)\theta\}\text{d}\theta \Big/ \int_0^{1/2} \theta \exp\{\log(x)\theta\}\text{d}\theta\\\end{align*} $$ that may seem to require the use of the incomplete Gamma function but which can be derived in closed form by integration by part: $$\int_0^{1/2} \theta^k \exp\{-\alpha\theta\}\text{d}\theta = \frac{-1}{\alpha}\left[\theta^k \exp\{-\alpha\theta\}\right]_0^{1/2} + \frac{k}{\alpha}\int_0^{1/2} \theta^{k-1} \exp\{-\alpha\theta\}\text{d}\theta$$ since $$\int_0^{1/2} \exp\{-\alpha\theta\}\text{d}\theta = \frac{1-\exp\{-\alpha/2\}}{\alpha}$$

Last, as indicated in my book, indeed, minimising in $\delta$ $$\int w(\theta)(\theta-\delta)^2 \pi(\theta|x) \text{d}\theta$$ is equivalent to minimising in $\delta$ $$\int w(\theta)(\theta-\delta)^2 \pi(\theta)f(x|\theta) \text{d}\theta$$ which itself is equivalent to minimising in $\delta$ $$\int (\theta-\delta)^2 w(\theta)\pi(\theta)f(x|\theta) \text{d}\theta$$ which amounts to replacing the original prior $\pi$ with a new prior $w(\theta)\pi(\theta)$ that needs to be renormalised into a density, that is, $$\pi_1(\theta)=w(\theta)\pi(\theta) \Big/\int w(\theta)\pi(\theta)\text{d}\theta$$

$\endgroup$
6
$\begingroup$

Your answer for the squared error loss part is wrong.

$$\pi(\theta|x) \propto f(x|\theta) \pi(\theta) = 2\theta x^{\theta-1}I_{(0,1/2)}(\theta). $$

This is a $Beta(\theta,1)$ distribution in $x$, not in $\theta$, and the random variable in the posterior is $\theta$. So your answer is incorrect, and the correct answer would be the posterior mean of that distribution.

For the second part,

(The prior for the weighted loss function is $\pi_1$ but you refer to it as $\pi$. I am switching notation back to $\pi_1$.)

Let $\pi'(\theta) = cw(\theta) \pi_1(\theta)$, where $c$ is a normalizing constant. You need to calculate

\begin{align*} \delta^{\pi_1}(x) & = \dfrac{E^{\pi_1}[w(\theta) \theta |x ]}{E^{\pi_1}[w(\theta|x)]}\\ & = \dfrac{\int w(\theta) \theta f(x|\theta) \pi_1(\theta)\, d\theta}{\int w(\theta) f(x|\theta)\pi_1(\theta)\, d\theta}\\ & = \dfrac{\int \theta f(x|\theta) \pi'(\theta) \,d\theta}{\int f(x|\theta) \pi'(\theta) \, d\theta}\\ & = E^{\pi'}[\theta|x] \end{align*}

Thus, for weighted least squares loss function, the theorem says that the Bayes estimate is the posterior mean with respect to a different prior. The prior being $$\pi'(\theta) \propto w(\theta) \pi_1(\theta). $$

The normalizing constant is $ \int_{\theta} w(\theta) \pi(\theta) d\theta = E_{\pi_1}[w(\theta)]$.

\begin{align*} E_{\pi_1}[w(\theta)] & = \int_{0}^{1/2} I_{0,1}(\theta) d(\theta) = \frac{1}{2}. \end{align*}

So the prior is $\pi'(\theta) = 2I_{(0,1/2)}(\theta)$. This is the same prior you had in the first question.

Thus the answer for the scenarios (whatever it is) will be the same. You can find the integral here. Although, it might be sufficient to right the form of the answer, and not complete the integral.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy