Logit Standard Error

Question

What is the SE of logit transformed variable $p$?

$logit = \log\frac{p}{1-p}$

where $p = \frac{n}{N}$

Is it:

$se = \sqrt{\frac{1}{n} + \frac{1}{N-n}}$

Answer 1

For a binomial random variable $X\sim \text{Bin}(n,p)$, the sample proportion $k/n$ is a consistent estimator of the probability parameter $p$, $\hat p = k/n$. We then have the asymptotic normality result

$$\sqrt n (\hat p -p) \xrightarrow{d} N(0, p(1-p))$$

Applying the Delta Theorem,

$$\sqrt n (g(\hat p) -g(p)) \xrightarrow{d} N(0, p(1-p)[g'(p)]^2)$$

Set $g(z) \equiv \ln(z/(1-z))$. Then

$$g'(z) = \frac {1-z}{z}\cdot \frac {1}{(1-z)^2} = \frac 1{z(1-z)}$$ Therefore

$$\sqrt n \left(\ln\frac{\hat p}{1-\hat p} -\ln\frac{ p}{1-p}\right) \xrightarrow{d} N\left(0, \frac 1{p(1-p)}\right)$$

In finite samples then we have the approximation

$$\ln\frac{\hat p}{1-\hat p} \sim_{\text{approx.}} N\left(\ln\frac{p}{1-p}, \frac 1{np(1-p)}\right)$$

To estimate the variance we use $\hat p =k/n$ instead of $p$ and we have

$$\hat Var \left(\ln\frac{\hat p}{1-\hat p}\right) = \frac {1}{n(k/n)(1-k/n)} = \frac 1k +\frac 1{n-k} $$

This is also the result obtained from the empirical inverted Hessian of the relevant log-likelihood.

Obviously, as Glen_b mentioned, if the observed proportion is $0$ or $1$ then this formula does not work.

ADDENDUM
Following conversation in the comments, I think we can describe the problem as follows:
The moments of the finite distribution of the logit transform, denote it $Z$ for brevity, are undefined in the sense that they contain an indeterminate form. For example, denoting $p_k$ the theoretical probability of the binomial taking the value $k$, we have

$$E(Z) = \sum_{k=0}^{n}p_k\ln[k/(n-k)]$$

$$= (1-p)^n\cdot \lim_{k\rightarrow 0}\ln[k/(n-k)] +...(\text{finite terms})...+ p^n\cdot \lim_{k\rightarrow n}\ln[k/(n-k)]$$

$$=(1-p)^n\left(-\infty - \ln[n]\right) +...+p^n\cdot \left(\ln[n] - (-\infty)\right)$$

$$=(1-p)^n\cdot (-\infty) +...+ p^n\cdot \infty$$

i.e. it contains a $-\infty +\infty$ expression.

If we now take the limit as $n\rightarrow \infty$, we will face

$$...=0\cdot (-\infty) +...+ 0 \cdot \infty$$

In general, the expression $0\cdot \pm \infty$ is also indeterminate, but from what I know, in measure theory $0\cdot \pm \infty$ is defined to be equal to $0$. So, where does that leave us?

Answer 2

$X\sim \text{Bin}(n,p)$

$p = X/n$

Since $p$ has a non-zero chance to be both 0 and 1, $E(\log(\frac{p}{1-p}))$, and also $\text{Var}(\log(\frac{p}{1-p}))$ are undefined.

If you want some other answer, you'll need to keep $p$ away from 0 and 1.