2
$\begingroup$

Suppose I have a proportion estimate and 95% CI: 0.39 (0.28, 0.58), how would I obtain the variance?

Typically 95% CIs are defined as $0.39 \pm 1.96\cdot SE = (0.28, 0.58)$. And suppose I know that the sample size $n = 100$. Since $SE = \frac{\sigma}{\sqrt{n}}$, I can easily solve for $\sigma^2$ with algebraic manipulation. However, I have two questions:

  1. How to deal with the asymmetric nature of this confidence interval? That is, the 2 estimates of $\sigma$ below are not consistent:

    • Lower bound: 0.$39-1.96\cdot \frac{\sigma}{10} = 0.28 \Rightarrow \sigma = 0.56$

    • Upper bound: $0.39 + 1.96\cdot \frac{\sigma}{10} = 0.58 \Rightarrow \sigma = 0.96$

  2. How to make sure that the proportion is $\in [0, 1]$?

    Suppose I take the larger value of $\sigma = 0.96$ just so the CI will be more conservative (wider), so my new lower bound is $0.39-1.96\cdot \frac {0.96}{10} = 0.21$ which is fine for this example, but there might be cases where taking the more conservative $\sigma$ will lead to a CI that is outside the bounds of $[0, 1]$

$\endgroup$
2
  • 4
    $\begingroup$ The CI has not been obtained using the approach you suggest, since that gives a symmetric interval. There are many CIs for binomial proportions; however you can estimate a standard error for the proportion from the proportion and $n$ $\endgroup$
    – Glen_b
    Apr 13 '17 at 7:58
  • 2
    $\begingroup$ If your proportions are close to zero or unity (for some meaning of close) you may want to transform them first anyway, work on the transformed scale, and then back-transform. $\endgroup$
    – mdewey
    Apr 13 '17 at 10:07
2
$\begingroup$

There are many ways to obtain confidence intervals. Generally, there are different corrections for different measures that all correct for some properties of the data, where the standard method may be inaccurate (e.g. Wherry's $R^2$ to correct for the amount of $IV$ in OLS regression). Similarly, in some cases there are methods to generate better CIs than the standard method.

Because the distribution of the correlation is not normal (not even symmetric) you have to use some trick. To give an extreme example consider a population correlation of $\rho=0.99$. Clearly, the distribution of all sample correlations $r$ around this $\rho$ will be (heavily) left-skewed.

Fishers Z transformation is good tool to use with correlations. You transform your correlation to follow an approximately normal distribution, calculate the CI, and transform the CI back for interpretation.

Let's start: You are given $r$ and $n$ the sample correlation and sample size.

First, you transform your $r$ to $r_z$ the Fisher transformed $r$.

$$r_z=\frac{1}{2} ln\left(\frac{1+r}{1-r}\right)$$

Now $r_z$ is approximately normal, $r_z\sim N\left(\rho_z, 1/\sqrt{n-3}\right)$. Therefore, we can easily apply the standard method to $r_z$.

The confidence interval will be

$$r_z \pm z* \frac{1}{\sqrt{n-3}}$$

Now you inverse the transformation:

$$r=\frac{e^{2 r_z}-1}{e^{2r_z}+1}$$

Apply this formula to the lower and upper bounds of $r_z$ and you will receive the CI for $r$.

Beyond your question: you can also apply standard null hypothesis significance testing to $r_z$. Though, if you are testing against $\rho=0.5$, you do not have to apply the transformation, because in this rare case $\rho$ already is normally distributed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.