Let $X\sim N(\theta,1)$ and consider 5 independent observations $X=(4.9,5.6,5.1,4.6,3.6)$. The prior probability that $\theta=4.01$ is $0.5$. The remain values of $\theta$ are given a prior with the density > $g(\theta)$.

a) Take $g(\theta)$ the density of $N(4.01,1)$ and test the hypothesis $$H_0:\theta=4.01\space vs\space H_1:\theta\neq 4.01$$

From what I learn to make a hypothesis test I need to find $$a_0=P(\theta\in\Theta_0|x)\qquad a_1=P(\theta\notin\Theta_0|x)$$ such that $$a_0+a_1=1$$ and reject $H_0$ if $$a_0<a_1$$ Cases when the null hypothesis is not a point I can solve, but in this case I have a few doubts.

From the notes that I take there is the theorem below

Theorem: For any prior $$\pi(\theta)=\pi_0\space \text{if}\space \theta=\theta_0$$ $$\pi(\theta)=\pi_1 g(\theta)\space\text{if}\space \theta\neq \theta_0$$ such that $$\int_{\theta\neq \theta_0}g(\theta)\text{d}\theta=1$$ then $$a_0=f(\theta|x)\geq \left[1+\frac{1-\pi_0}{\pi_0}\frac{r(x)}{f(x|\theta_0)}\right]^{-1}$$ where $$r(x)=\sup_{\theta\neq\theta_0}f(x|\theta)$$ usually $$r(x)=f(x|\hat{\theta})$$

In this case $\hat{\theta}=\overline{X}$ but the distribution of $$f(x|\overline{X})$$ doesn't make sense to me, in one example that I look they take $$f(\overline{x}|\hat{\theta})$$ but I don't understood the logic.

I need to use the distribution of the likelihood estimator supposing that $\theta=\hat{\theta}$?

If someone can give me a explanation with details on how it works I really appreciate.

  • 1
    $\begingroup$ Which textbook do you mean by "the textbook"? Note that if the question comes from the textbook you should give a proper reference (author, year, title, edition if later than first edition, publisher) $\endgroup$ – Glen_b Apr 28 '16 at 23:20
  • 1
    $\begingroup$ @Glen_b The question don't comes from a textbook, I just use the book as a supplement for the notes. The question comes from theses notes that I found. $\endgroup$ – user72621 Apr 29 '16 at 0:03

There is a mistake in your notes: if you have a point mass at $\theta_0$ with a prior$$\pi(\theta)=\pi_0\delta_{\theta_0}(\theta)+\pi_1 g(\theta)$$where $\delta_{\theta_0}$ denotes the Dirac point mass at $\theta_0$, the posterior distribution on $\theta$ given $x\sim f(x|\theta)$ is$$\pi(\theta|x)\propto f(x|\theta) \{\pi_0\delta_{\theta_0}(\theta)+\pi_1 g(\theta)\}=\pi_0f(x|\theta_0)\delta_{\theta_0}(\theta)+\pi_1 f(x|\theta)g(\theta)$$which has as normalising constant$$\begin{align*} \int \pi_0f(x|\theta_0)\delta_{\theta_0}(\theta)+\pi_1 f(x|\theta)g(\theta)\text{d}\theta&=\pi_0f(x|\theta_0)+\pi_1\overbrace{\int_{\theta\ne\theta_0}f(x|\theta)g(\theta)\text{d}\theta}^{\text{denote by }r(x)}\\ &=\pi_0f(x|\theta_0)+\pi_1\,r(x) \end{align*}$$ Hence, $$a_0=\pi(\theta=\theta_0|x)=\dfrac{\pi_0 f(x|\theta_0)}{\pi_0f(x|\theta_0)+\pi_1\,r(x)}=\left[1+\frac{\pi_1\,r(x)}{\pi_0 f(x|\theta_0)}\right]^{-1}$$ In this solution $r(x)$ is the marginal density of $x$, not a supremum and not a plug-in quantity. For your example, since $\bar{X}\sim\mathcal{N}(\theta,1/5)$, if one assumes a prior like $\theta\sim\mathcal{N}(\theta_0,1)$ [but any other choice is as acceptable in this framework], then $$r(x)=r(\bar{x})=\int_{\mathbb{R}} \sqrt{5}\varphi(\{\bar{x}-\theta\}\sqrt{5})\varphi(\theta-\theta_0)\text{d}\theta=\varphi(\{\bar{x}-\theta_0\}/\sqrt{6/5})/\sqrt{6/5}$$the density of a Gaussian distribution $\mathcal{N}(\theta_0,1+\frac{1}{5})$.

Remark: An explanation for my choice of a $\theta\sim\mathcal{N}(\theta_0,1)$ prior is that (a) I have no idea what the problem is all about, hence no prior information; (b) the value $\theta_0$ is a special value since it is tested and a possible value for $\theta$ as otherwise it would not be tested, so I centre my prior around this value; (c) the choice of a unit variance means that I weight my prior choice as much as a single observation in the sample, which is a reasonable scale if I do not have other items of prior information. An alternative would be to chose a Cauchy $C(\theta_0,1)$ as being less narrow than the normal and it would lead to a different numerical and formal answer, both being acceptable within the Bayesian paradigm.

  • $\begingroup$ Is there some criterior for choice $g(\theta)$ ? $\endgroup$ – user72621 Apr 29 '16 at 22:42
  • $\begingroup$ No, there is no such thing as a default prior. $\endgroup$ – Xi'an Apr 30 '16 at 9:41

You are tasked with finding the posterior probabilities of two hypotheses and picking the hypothesis with the higher posterior probability. In this case it can be done with a straightforward application of Bayes' theorem. You will need the discrete form of Bayes' theorem: $$P(H_i|X) = \dfrac{P(H_i)P(X|H_i)}{\sum_{k=1}^K P(H_k)P(X|H_k)}$$ that applies when there are $K$ competing hypotheses, one of which is $H_i$. You have $i = {0, 1}$ for a total of $K = 2$ hypotheses.

We have the following information (from the stated problem):

  • $X\sim N(\theta, 1)$ [Data are normal with SD known]
  • Observations: X=(4.9,5.6,5.1,4.6,3.6) [mean(X) = 4.76]
  • $P(H_0) = .5 = P(H_1)$ [Prior probabilities of the hypotheses]
  • Under $H_1: g(\theta) \sim N(4.01, 1)$ [Prior probability distribution under $H_1$]
  • Under $H_0: \theta = \theta_0 = 4.01$ [Point mass under $H_0$]

So looking back at Bayes' theorem, you can see you have both $P(H_1)$ and $P(H_0)$, as well as X (the data), and you need to find $P(X\mid H_1)$ and $P(X\mid H_0)$.

$P(X\mid H_0)$ is $f(x\mid \theta=\theta_0)$.

$P(X\mid H_1)$ is $\int_\Theta g(\theta)f(x\mid\theta) d\theta$.

Then plug these terms into Bayes' theorem along with the prior probability terms and you're all set. Do you know how to solve these last two terms?

(The theorem you provide gives a way to find the lower limit to the posterior probability of $H_0$ for a given dataset, but it doesn't really help you to solve your problem.)

  • $\begingroup$ I don't understood your argument. Why I assume that in $H_1$ $g(\theta)\sim N(4.01,1)$ if the hypothesis is $\theta\neq 4.01$?And how you solve the last two terms that you said? $\endgroup$ – user72621 Apr 29 '16 at 0:12
  • $\begingroup$ Your problem states it that way: "The remain values of $\theta$ are given the density of g(θ)." $\endgroup$ – Alexander Etz Apr 29 '16 at 0:15
  • $\begingroup$ That's right, but how I solve $P(X|H_0)$? Because the posterior is normal. $\endgroup$ – user72621 Apr 29 '16 at 0:19
  • $\begingroup$ You can solve $P(X\mid H_0)$ by calculating the probability density for the data's mean (i.e., 4.76) from a normal distribution that has mean = 4.01 and variance = 1. $\endgroup$ – Alexander Etz Apr 29 '16 at 0:24
  • $\begingroup$ I do not understand how this calculation is made. $\endgroup$ – user72621 Apr 29 '16 at 0:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy