5
$\begingroup$

What is the SE of logit transformed variable $p$?

$logit = \log\frac{p}{1-p}$

where $p = \frac{n}{N}$

Is it:

$se = \sqrt{\frac{1}{n} + \frac{1}{N-n}}$

$\endgroup$
4
  • $\begingroup$ Usually "$p$" denotes a (usually unknown) constant (which does not have a variance, and so neither does any function of it). So I suspect that here "$p$" actually stands for $\hat p$, a random variable that estimates the true $p$. is that so? Please clarify. In general, the moments of non-linear functions of random variables are determined as an approximation or as an asymptotic result, using the asymptotic distribution of $\hat p$ (normal?), and the Delta Theorem (or "method"). So how is $\hat p$ defined? $\endgroup$ Oct 9, 2014 at 16:21
  • $\begingroup$ p stands for a sample proportion, which presumably should be a binomially distributed variable. $\endgroup$ Oct 10, 2014 at 8:03
  • $\begingroup$ Sorry, I had it wrong, now it's what I was thinking. $\endgroup$ Oct 10, 2014 at 13:40
  • $\begingroup$ @Germaniawerks, I believe Alecos Papadopoulos' answer is what you were looking for. How about marking it as accepted? $\endgroup$ Jun 24, 2017 at 2:49

2 Answers 2

6
$\begingroup$

For a binomial random variable $X\sim \text{Bin}(n,p)$, the sample proportion $k/n$ is a consistent estimator of the probability parameter $p$, $\hat p = k/n$. We then have the asymptotic normality result

$$\sqrt n (\hat p -p) \xrightarrow{d} N(0, p(1-p))$$

Applying the Delta Theorem,

$$\sqrt n (g(\hat p) -g(p)) \xrightarrow{d} N(0, p(1-p)[g'(p)]^2)$$

Set $g(z) \equiv \ln(z/(1-z))$. Then

$$g'(z) = \frac {1-z}{z}\cdot \frac {1}{(1-z)^2} = \frac 1{z(1-z)}$$ Therefore

$$\sqrt n \left(\ln\frac{\hat p}{1-\hat p} -\ln\frac{ p}{1-p}\right) \xrightarrow{d} N\left(0, \frac 1{p(1-p)}\right)$$

In finite samples then we have the approximation

$$\ln\frac{\hat p}{1-\hat p} \sim_{\text{approx.}} N\left(\ln\frac{p}{1-p}, \frac 1{np(1-p)}\right)$$

To estimate the variance we use $\hat p =k/n$ instead of $p$ and we have

$$\hat Var \left(\ln\frac{\hat p}{1-\hat p}\right) = \frac {1}{n(k/n)(1-k/n)} = \frac 1k +\frac 1{n-k} $$

This is also the result obtained from the empirical inverted Hessian of the relevant log-likelihood.

Obviously, as Glen_b mentioned, if the observed proportion is $0$ or $1$ then this formula does not work.

ADDENDUM
Following conversation in the comments, I think we can describe the problem as follows:
The moments of the finite distribution of the logit transform, denote it $Z$ for brevity, are undefined in the sense that they contain an indeterminate form. For example, denoting $p_k$ the theoretical probability of the binomial taking the value $k$, we have

$$E(Z) = \sum_{k=0}^{n}p_k\ln[k/(n-k)]$$

$$= (1-p)^n\cdot \lim_{k\rightarrow 0}\ln[k/(n-k)] +...(\text{finite terms})...+ p^n\cdot \lim_{k\rightarrow n}\ln[k/(n-k)]$$

$$=(1-p)^n\left(-\infty - \ln[n]\right) +...+p^n\cdot \left(\ln[n] - (-\infty)\right)$$

$$=(1-p)^n\cdot (-\infty) +...+ p^n\cdot \infty$$

i.e. it contains a $-\infty +\infty$ expression.

If we now take the limit as $n\rightarrow \infty$, we will face

$$...=0\cdot (-\infty) +...+ 0 \cdot \infty$$

In general, the expression $0\cdot \pm \infty$ is also indeterminate, but from what I know, in measure theory $0\cdot \pm \infty$ is defined to be equal to $0$. So, where does that leave us?

$\endgroup$
5
  • $\begingroup$ Actually, even if the observed proportion can be 0 or 1 with positive probability on both, we're really estimating a variance that doesn't exist. (On the other hand if we say they can't be 0 or 1, we're not dealing with a binomial. Saying something correct is tricky ... and I've fallen into most of the traps, one way or another.) $\endgroup$
    – Glen_b
    Oct 13, 2014 at 4:03
  • $\begingroup$ @Glen_b I am not sure I follow you here: We are estimating the variance of the asymptotic distribution, which does exist, irrespective of the observed proportion's value no? It was in this spirit that I wrote "the formula doesn't work" instead of "the variance doesn't exist". $\endgroup$ Oct 13, 2014 at 8:53
  • $\begingroup$ I find myself in two minds on this -- the limit of something that's undefined at every finite $n$ concerns me. On the other hand, if you do bound it away from 0 and 1, the results tend to work pretty well. $\endgroup$
    – Glen_b
    Oct 13, 2014 at 9:43
  • $\begingroup$ @Glen_b I added in the question a mathematical exposition of the issue, as I understand it. $\endgroup$ Oct 13, 2014 at 10:29
  • $\begingroup$ I've just been looking at "An Introduction to Measure Theory", Terry Tao (ed.). on google books, page xi -- which seems to suggest that there's two different ways you can do measure theory - and if you're taking the $0\cdot \infty = 0$ route (non negative theory), then you don't have $lim_{n\to\infty}\frac{1}{n}\cdot\infty=0\cdot\infty$ (it's not downward continuous). But I expect you know a good deal more of this stuff than I do. $\endgroup$
    – Glen_b
    Oct 13, 2014 at 11:06
1
$\begingroup$

$X\sim \text{Bin}(n,p)$

$p = X/n$

Since $p$ has a non-zero chance to be both 0 and 1, $E(\log(\frac{p}{1-p}))$, and also $\text{Var}(\log(\frac{p}{1-p}))$ are undefined.

If you want some other answer, you'll need to keep $p$ away from 0 and 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.