Identification of asymptotic distribution of a function of the MLE

Question

Let $X_1, X_2, \dots, X_n$ be Bernoulli$(\theta)$ and let $\hat{\theta}$ be the MLE of $\theta$. I am attempting to identify the asymptotic distribution of the odds ratio. I believe that I understand how to identify the asymptotic variance of a function $\tau(\hat{\theta})$ (see equation 10.1.7 of Casella and Berger), but it is unclear (to me) how to identify the mean.

For example, take the odds ratio to be $\tau(\theta)$:

$$\tau(\theta) = \frac{\hat{\theta}}{1-\hat{\theta}}$$

It can be shown (Casella Berger Ex 10.1.14) that asymptotic variance thereof is:

$$\frac{\hat{\theta}}{n(1-\hat{\theta})^3}$$

Intuitively, I expect the odds ratio parameterized by the MLE to be the limiting expected value, but how do I formally make this assertion?

Answer 1

I think that the delta method is a good way to go here (and if you're working out of C&B you'll probably need it for other problems). Let $X_n$ be a sequence of random variables satisfying $$ \sqrt n (X_n - \theta) \rightarrow_d \mathcal N(0, \sigma^2). $$

Let $g$ be a function where $g'(\theta)$ exists and is non-zero. Then $$ \sqrt n (g(X_n) - g(\theta)) \rightarrow_d \mathcal N(0, g'(\theta)^2 \sigma^2). $$

We have $X_1, X_2, \dots \overset{\textrm{iid}}{\sim} \ Bern(\theta) \implies \hat \theta = \bar X_n$ (I'm assuming iid). $\mathbb E(X_i) = \theta < \infty$ and $\operatorname{Var}(X_i) = \theta(1-\theta) < \infty$ so by the CLT $$ \sqrt n (\hat \theta_n - \theta) \rightarrow_d \mathcal N(0, \theta(1-\theta)). $$

This means that we can take $g(x) = {x \over 1-x}$ and apply the delta method: $$ g'(x) = \frac{1}{(1-x)^2} $$

therefore $$ \sqrt n \left(\frac{\hat \theta}{1 - \hat \theta} - \frac{\theta}{1-\theta}\right) \rightarrow_d \mathcal N\left(0, \frac{\theta(1-\theta)}{(1-\theta)^4}\right) $$

$$ =_d \mathcal N \left(0, \frac{\theta}{(1-\theta)^3} \right). $$

Answer 2

There are a number of ways to approach it, but one thing you might consider is the possibility of expanding $(1-\hat{\theta})^{-1}$ in a power series. However, you'll have to make sure everything converges appropriately as you carry out the operations you need (e.g. you need to be sure it's okay to do the expansion in the first place and to take the expectation inside the expanded series and taking limits and so on).

Take care if you do go about it this way. (There may be other, better, approaches.)

Another possibility might be to consider attempting to apply Slutsky's theorem.