# Logit Standard Error

What is the SE of logit transformed variable $p$?

$logit = \log\frac{p}{1-p}$

where $p = \frac{n}{N}$

Is it:

$se = \sqrt{\frac{1}{n} + \frac{1}{N-n}}$

• Usually "$p$" denotes a (usually unknown) constant (which does not have a variance, and so neither does any function of it). So I suspect that here "$p$" actually stands for $\hat p$, a random variable that estimates the true $p$. is that so? Please clarify. In general, the moments of non-linear functions of random variables are determined as an approximation or as an asymptotic result, using the asymptotic distribution of $\hat p$ (normal?), and the Delta Theorem (or "method"). So how is $\hat p$ defined? Commented Oct 9, 2014 at 16:21
• p stands for a sample proportion, which presumably should be a binomially distributed variable. Commented Oct 10, 2014 at 8:03
• Sorry, I had it wrong, now it's what I was thinking. Commented Oct 10, 2014 at 13:40
• @Germaniawerks, I believe Alecos Papadopoulos' answer is what you were looking for. How about marking it as accepted? Commented Jun 24, 2017 at 2:49

For a binomial random variable $X\sim \text{Bin}(n,p)$, the sample proportion $k/n$ is a consistent estimator of the probability parameter $p$, $\hat p = k/n$. We then have the asymptotic normality result

$$\sqrt n (\hat p -p) \xrightarrow{d} N(0, p(1-p))$$

Applying the Delta Theorem,

$$\sqrt n (g(\hat p) -g(p)) \xrightarrow{d} N(0, p(1-p)[g'(p)]^2)$$

Set $g(z) \equiv \ln(z/(1-z))$. Then

$$g'(z) = \frac {1-z}{z}\cdot \frac {1}{(1-z)^2} = \frac 1{z(1-z)}$$ Therefore

$$\sqrt n \left(\ln\frac{\hat p}{1-\hat p} -\ln\frac{ p}{1-p}\right) \xrightarrow{d} N\left(0, \frac 1{p(1-p)}\right)$$

In finite samples then we have the approximation

$$\ln\frac{\hat p}{1-\hat p} \sim_{\text{approx.}} N\left(\ln\frac{p}{1-p}, \frac 1{np(1-p)}\right)$$

To estimate the variance we use $\hat p =k/n$ instead of $p$ and we have

$$\hat Var \left(\ln\frac{\hat p}{1-\hat p}\right) = \frac {1}{n(k/n)(1-k/n)} = \frac 1k +\frac 1{n-k}$$

This is also the result obtained from the empirical inverted Hessian of the relevant log-likelihood.

Obviously, as Glen_b mentioned, if the observed proportion is $0$ or $1$ then this formula does not work.

Following conversation in the comments, I think we can describe the problem as follows:
The moments of the finite distribution of the logit transform, denote it $Z$ for brevity, are undefined in the sense that they contain an indeterminate form. For example, denoting $p_k$ the theoretical probability of the binomial taking the value $k$, we have

$$E(Z) = \sum_{k=0}^{n}p_k\ln[k/(n-k)]$$

$$= (1-p)^n\cdot \lim_{k\rightarrow 0}\ln[k/(n-k)] +...(\text{finite terms})...+ p^n\cdot \lim_{k\rightarrow n}\ln[k/(n-k)]$$

$$=(1-p)^n\left(-\infty - \ln[n]\right) +...+p^n\cdot \left(\ln[n] - (-\infty)\right)$$

$$=(1-p)^n\cdot (-\infty) +...+ p^n\cdot \infty$$

i.e. it contains a $-\infty +\infty$ expression.

If we now take the limit as $n\rightarrow \infty$, we will face

$$...=0\cdot (-\infty) +...+ 0 \cdot \infty$$

In general, the expression $0\cdot \pm \infty$ is also indeterminate, but from what I know, in measure theory $0\cdot \pm \infty$ is defined to be equal to $0$. So, where does that leave us?

• Actually, even if the observed proportion can be 0 or 1 with positive probability on both, we're really estimating a variance that doesn't exist. (On the other hand if we say they can't be 0 or 1, we're not dealing with a binomial. Saying something correct is tricky ... and I've fallen into most of the traps, one way or another.) Commented Oct 13, 2014 at 4:03
• @Glen_b I am not sure I follow you here: We are estimating the variance of the asymptotic distribution, which does exist, irrespective of the observed proportion's value no? It was in this spirit that I wrote "the formula doesn't work" instead of "the variance doesn't exist". Commented Oct 13, 2014 at 8:53
• I find myself in two minds on this -- the limit of something that's undefined at every finite $n$ concerns me. On the other hand, if you do bound it away from 0 and 1, the results tend to work pretty well. Commented Oct 13, 2014 at 9:43
• @Glen_b I added in the question a mathematical exposition of the issue, as I understand it. Commented Oct 13, 2014 at 10:29
• I've just been looking at "An Introduction to Measure Theory", Terry Tao (ed.). on google books, page xi -- which seems to suggest that there's two different ways you can do measure theory - and if you're taking the $0\cdot \infty = 0$ route (non negative theory), then you don't have $lim_{n\to\infty}\frac{1}{n}\cdot\infty=0\cdot\infty$ (it's not downward continuous). But I expect you know a good deal more of this stuff than I do. Commented Oct 13, 2014 at 11:06

$X\sim \text{Bin}(n,p)$

$p = X/n$

Since $p$ has a non-zero chance to be both 0 and 1, $E(\log(\frac{p}{1-p}))$, and also $\text{Var}(\log(\frac{p}{1-p}))$ are undefined.

If you want some other answer, you'll need to keep $p$ away from 0 and 1.