8
$\begingroup$

I have a three part question:

1) If I have a $Cauchy(\theta, 1)$ with density:

$p(x-\theta) = \frac{1}{\pi\{1+{(x-\theta)^2}\}}$

and $x_1, ..., x_n$ forms i.i.d sample. I see on Wiki that this will make a likelihood function of:

$l(\theta) = -nlog\pi - \sum_{i=1}^{n} log(1+{(x_i-\theta)^2})$

But how is that derived?

2) What derivative rules would I use to show that:

$l'(\theta) = -2 \sum_{i=1}^{n} \frac{x_i-\theta}{1+(x_i-\theta)^2}$

(I feel this is probably straight-forward, but I am missing it)

3) How would I obtain that the Fisher information is equal to $I(\theta)=\frac{n}{2}$?

I started by:

$\frac{\partial^2f(x;\theta)}{2\theta^2} = \frac{8(x-\theta)^2}{\pi[1+(x-\theta)^2]^3} - \frac{2}{\pi[1+(x-\theta)^2]^2}$

I think that I next need to find the integral of:

$I(\theta) = -E[\frac{\partial^2f(x;\theta)}{2\theta^2}]$

But I cannot get this last step of reducing this information to $\frac{n}{2}$.

$\endgroup$
8
  • 1
    $\begingroup$ See wwwf.imperial.ac.uk/~das01/MyWeb/M3S3/Handouts/… Nothing special - just integration. $\endgroup$ Apr 6, 2015 at 14:40
  • 1
    $\begingroup$ 3) Fisher information is defined as$$I(\theta) = -\mathbb{E}_\theta\left[\frac{\partial^2 \log f(X;\theta)}{\partial\theta^2}\right]$$ $\endgroup$
    – Xi'an
    Apr 6, 2015 at 14:41
  • $\begingroup$ @Alexey: Thanks, that sure is a lot of integration. I am pretty familiar with tan/cos/sin integration. However, I am unsure how to integrate my specific case, where I have $l(\theta) = -nlog\pi - \sum_{i=1}^{n} log(1+{(x_i-\theta)^2})$. Could I begin by saying $I(\theta) = 2n \int_{-\infty}^\infty \frac{1-(x-\theta)^2}{(1+(x-\theta)^2)^2} \frac{1}{\pi} \frac{1}{1+(x-\theta)^2}dx$ ? I am not sure how to deal with the summation of n in the easiest way possible. $\endgroup$
    – user84756
    Apr 6, 2015 at 15:39
  • $\begingroup$ To clarify, my question is how to get the n constant out of the way before all the integration! Thanks again. $\endgroup$
    – user84756
    Apr 6, 2015 at 15:45
  • 1
    $\begingroup$ Could you please correct the mistakes in the question, i.e., $l^\prime(\theta)=-2...$ which should be $l^\prime(\theta)=+2...$ and the wrong resolution you copied from this forum? $\endgroup$
    – Xi'an
    Apr 7, 2015 at 7:34

1 Answer 1

8
$\begingroup$

The Fisher information for one observation is given by\begin{align*}I(\theta) &= -\mathbb{E}_\theta\left[\frac{\partial^2 \log f(X;\theta)}{\partial\theta^2}\right]\\ &=\mathbb{E}_\theta\left[ \frac{\partial^2 \log \{1+(X-\theta)^2\}}{\partial\theta^2}\right]\\ &=2\mathbb{E}_\theta\left[ -\frac{\partial }{\partial\theta}\frac{(X-\theta)}{1+(X-\theta)^2}\right]\\ &=2\mathbb{E}_\theta\left[\frac{1}{1+(X-\theta)^2}-\frac{2(X-\theta)^2}{[1+(X-\theta)^2]^2}\right]\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+(x-\theta)^2]^2}-\frac{2(x-\theta)^2}{[1+(x-\theta)^2]^3} \text{d}x\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+x^2]^2}-\frac{2x^2}{[1+x^2]^3} \text{d}x\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+x^2]^2}-\frac{2}{[1+x^2]^2}+\frac{2}{[1+x^2]^3} \text{d}x\\ &= \frac{2}{\pi}\int_\mathbb{R} \frac{-1}{[1+x^2]^2}+\frac{2}{[1+x^2]^3} \text{d}x \end{align*} because the integral (and the information) is translation invariant.

Now it is easy to establish a recurrence relation on$$I_k=\int_\mathbb{R} \frac{1}{[1+x^2]^k}\text{d}x$$Indeed \begin{align*} I_k &= \int_\mathbb{R} \frac{1+x^2}{[1+x^2]^{k+1}}\text{d}x\\ &= I_{k+1} + \int_\mathbb{R} \frac{2kx}{[1+x^2]^{k+1}}\frac{x}{2k}\text{d}x\\ &= I_{k+1} + \frac{1}{2k} \int_\mathbb{R} \frac{1}{[1+x^2]^{k}}\text{d}x = I_{k+1} + \frac{1}{2k} I_k \end{align*} by an integration by parts. Hence $$I_1=\pi\quad\text{and}\quad I_{k+1}=\frac{2k-1}{2k}I_k\quad k>1$$ which implies $$I_1=\pi\quad I_2=\frac{\pi}{2}\quad I_3=\frac{3\pi}{8}$$ and which leads to the Fisher information: $$I(\theta)=\frac{2}{\pi}\left\{-I_2+2I_3\right\}=\frac{2}{\pi}\left\{\frac{-\pi}{2}+\frac{3\pi}{4}\right\}=\frac{1}{2}$$

$\endgroup$
3
  • 4
    $\begingroup$ +1. You could also note that differentiating both sides of $$\int_\mathbb{R}\frac{dx}{t+x^2}=\pi t^{-1/2}$$ twice in succession with respect to $t$ and setting $t=1$ easily yields the answer. $\endgroup$
    – whuber
    Apr 6, 2015 at 21:32
  • $\begingroup$ (+1) Even nicer! $\endgroup$
    – Xi'an
    Apr 8, 2015 at 19:30
  • $\begingroup$ Working with $I(\theta)=\text{E}_{\theta}\left[\frac{\partial}{\partial\theta}\ln f_{\theta}(X_1)\right]^2$ makes calculations simpler here as we end up with a Beta integral of the second kind. $\endgroup$ Aug 10, 2018 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.