Let $y_1, ... y_T$ be a random sample of T = 100 observations on iid Bernoulli distributed random variables $Y_t$ that represent individual decision making where $y_t$ = 1 with probability θ and $y_t$ = 0 with probability (1 - θ).

Τhe density is given by $f(y_t ; θ) = θ^{y_t}(1 - θ)^{1 - y_t}$.

The MLE estimator of of theta is $\hat{θ}$ = $\frac{\sum y_t}{T}$ = $\bar{y}$.

The variance of the MLE estimator is $\hat{σ^2}$ = $\frac{θ(1 - θ)}{Τ}$

I want to use the asymptotic normal approximation to derive an asymptotically valid interval for $\frac{θ}{1 - θ}$ at 95%.

I think this is related to the delta method where if X is a random variable then variance of g(X) is given as $σ^2$  times $g'(X)^2$.

If I'm right then $g'(X)^2$ = $\frac{1}{(1 - θ)^4}$.

Τhen we have $\frac{θ(1 - θ)}{T}$ as the estimator of the $σ^2$ and this is multiplied by $g'(X)^2$ = $\frac{1}{(1 - θ)^4}$

Then the asymptotic variance of is $\frac{θ(1 - θ)}{Τ}$ times $\frac{1}{(1 - θ)^4}$ = $\frac{θ}{Τ(1 - θ)^3}$

Is this correct?

If so then the 95% CI would be $\hat{θ} \pm 1.96 \sqrt{\frac{θ}{T(1 - θ)^3}}$

Or (again assuming the variance is correct) would the CI be

$\frac{\hat{θ}}{1 - \hat{θ}} \pm 1.96 \sqrt{\frac{θ}{T(1 - θ)^3}}$ ?

  • $\begingroup$ When I first wrote it I used the Binomial distribution and the result had a T in the numerator. I thought perhaps it should be the Bernoulli distribution since we are looking at each $y_t$ as a Bernoulli. $\endgroup$ – MHall Oct 21 '18 at 0:07
  • $\begingroup$ After density, need to derive how to estimate $θ$. derived the variance of this estimate. Then consider the function of $θ$. $\endgroup$ – user158565 Oct 21 '18 at 0:40
  • $\begingroup$ I updated the question to include the derivations of the estimator of θ and the variance of that estimate. $\endgroup$ – MHall Oct 21 '18 at 2:55
  • $\begingroup$ I suggest studying the Wilson confidence interval for proportions. $\endgroup$ – Frank Harrell Oct 21 '18 at 3:12
  • $\begingroup$ I rewrite the second half in answer. But $g(θ)=\frac θ{1−θ}$ is monotonic function of $θ$, so get the CI for $θ$ first and convert CI by g function is good method. $\endgroup$ – user158565 Oct 21 '18 at 3:29

By the delta method, if $X$ is a random variable with $\mathrm{Var}(X) = \sigma^2$ then $\mathrm{Var}(f(X)) = \sigma^2 [g′(X)]^2$.

For $g(θ) = \frac θ{1−θ}$, we can estimate $g(θ)$ by $\hat{g(θ)} = \frac {\hat θ}{1−\hat θ}$

Then by the delta method, $\mathrm{Var}(\hat{g(θ)}) = \mathrm{Var}(\hat θ)[g'(θ)]^2 =\frac{θ(1-θ)}T\frac 1{(1-θ)^4} = \frac θ {T(1-θ)^3} $

The 95% CI of $g(θ)$ can be get by $\frac {\hat θ}{1−\hat θ} \pm 1.96\sqrt{\frac {\hat {θ}} {T(1-\hat {θ})^3}}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.