5
$\begingroup$

I have a three part question:

1) If I have a $Cauchy(\theta, 1)$ with density:

$p(x-\theta) = \frac{1}{\pi\{1+{(x-\theta)^2}\}}$

and $x_1, ..., x_n$ forms i.i.d sample. I see on Wiki that this will make a likelihood function of:

$l(\theta) = -nlog\pi - \sum_{i=1}^{n} log(1+{(x_i-\theta)^2})$

But how is that derived?

2) What derivative rules would I use to show that:

$l'(\theta) = -2 \sum_{i=1}^{n} \frac{x_i-\theta}{1+(x_i-\theta)^2}$

(I feel this is probably straight-forward, but I am missing it)

3) How would I obtain that the Fisher information is equal to $I(\theta)=\frac{n}{2}$?

I started by:

$\frac{\partial^2f(x;\theta)}{2\theta^2} = \frac{8(x-\theta)^2}{\pi[1+(x-\theta)^2]^3} - \frac{2}{\pi[1+(x-\theta)^2]^2}$

I think that I next need to find the integral of:

$I(\theta) = -E[\frac{\partial^2f(x;\theta)}{2\theta^2}]$

But I cannot get this last step of reducing this information to $\frac{n}{2}$.

$\endgroup$
  • 1
    $\begingroup$ See wwwf.imperial.ac.uk/~das01/MyWeb/M3S3/Handouts/… Nothing special - just integration. $\endgroup$ – Alexey Zaytsev Apr 6 '15 at 14:40
  • 1
    $\begingroup$ 3) Fisher information is defined as$$I(\theta) = -\mathbb{E}_\theta\left[\frac{\partial^2 \log f(X;\theta)}{\partial\theta^2}\right]$$ $\endgroup$ – Xi'an Apr 6 '15 at 14:41
  • $\begingroup$ @Alexey: Thanks, that sure is a lot of integration. I am pretty familiar with tan/cos/sin integration. However, I am unsure how to integrate my specific case, where I have $l(\theta) = -nlog\pi - \sum_{i=1}^{n} log(1+{(x_i-\theta)^2})$. Could I begin by saying $I(\theta) = 2n \int_{-\infty}^\infty \frac{1-(x-\theta)^2}{(1+(x-\theta)^2)^2} \frac{1}{\pi} \frac{1}{1+(x-\theta)^2}dx$ ? I am not sure how to deal with the summation of n in the easiest way possible. $\endgroup$ – user84756 Apr 6 '15 at 15:39
  • $\begingroup$ To clarify, my question is how to get the n constant out of the way before all the integration! Thanks again. $\endgroup$ – user84756 Apr 6 '15 at 15:45
  • 1
    $\begingroup$ Could you please correct the mistakes in the question, i.e., $l^\prime(\theta)=-2...$ which should be $l^\prime(\theta)=+2...$ and the wrong resolution you copied from this forum? $\endgroup$ – Xi'an Apr 7 '15 at 7:34
6
$\begingroup$

The Fisher information for one observation is given by\begin{align*}I(\theta) &= -\mathbb{E}_\theta\left[\frac{\partial^2 \log f(X;\theta)}{\partial\theta^2}\right]\\ &=\mathbb{E}_\theta\left[ \frac{\partial^2 \log \{1+(X-\theta)^2\}}{\partial\theta^2}\right]\\ &=2\mathbb{E}_\theta\left[ -\frac{\partial }{\partial\theta}\frac{(X-\theta)}{1+(X-\theta)^2}\right]\\ &=2\mathbb{E}_\theta\left[\frac{1}{1+(X-\theta)^2}-\frac{2(X-\theta)^2}{[1+(X-\theta)^2]^2}\right]\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+(x-\theta)^2]^2}-\frac{2(x-\theta)^2}{[1+(x-\theta)^2]^3} \text{d}x\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+x^2]^2}-\frac{2x^2}{[1+x^2]^3} \text{d}x\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+x^2]^2}-\frac{2}{[1+x^2]^2}+\frac{2}{[1+x^2]^3} \text{d}x\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{-1}{[1+x^2]^2}+\frac{2}{[1+x^2]^3} \text{d}x \end{align*} because the integral (and the information) is translation invariant.

Now it is easy to establish a recurrence relation on$$I_k=\int_\mathbb{R} \frac{1}{[1+x^2]^k}\text{d}x$$Indeed \begin{align*} I_k &= \int_\mathbb{R} \frac{1+x^2}{[1+x^2]^{k+1}}\text{d}x\\ &= I_{k+1} + \int_\mathbb{R} \frac{2kx}{[1+x^2]^{k+1}}\frac{x}{2k}\text{d}x\\ &= I_{k+1} + \frac{1}{2k} \int_\mathbb{R} \frac{1}{[1+x^2]^{k}}\text{d}x = I_{k+1} + \frac{1}{2k} I_k \end{align*} by an integration by parts. Hence $$I_1=\pi\quad\text{and}\quad I_{k+1}=\frac{2k-1}{2k}I_k\quad k>1$$ which implies $$I_1=\pi\quad I_2=\frac{\pi}{2}\quad I_3=\frac{3\pi}{8}$$ and which leads to the Fisher information: $$I(\theta)=\frac{2}{\pi}\left\{-I_2+2I_3\right\}=\frac{2}{\pi}\left\{\frac{-\pi}{2}+\frac{3\pi}{4}\right\}=\frac{1}{2}$$

$\endgroup$
  • 2
    $\begingroup$ +1. You could also note that differentiating both sides of $$\int_\mathbb{R}\frac{dx}{t+x^2}=\pi t^{-1/2}$$ twice in succession with respect to $t$ and setting $t=1$ easily yields the answer. $\endgroup$ – whuber Apr 6 '15 at 21:32
  • $\begingroup$ (+1) Even nicer! $\endgroup$ – Xi'an Apr 8 '15 at 19:30
  • $\begingroup$ Working with $I(\theta)=\text{E}_{\theta}\left[\frac{\partial}{\partial\theta}\ln f_{\theta}(X_1)\right]^2$ makes calculations simpler here as we end up with a Beta integral of the second kind. $\endgroup$ – StubbornAtom Aug 10 '18 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.