I have a population in which some have an event A and some other don't. Event A is actually my target class. I also have a set of variables/features for my population which I can use in a modeling (supervised learning) setting. Let's say one of the features/variables is age. What I'd like to find is the impact of age on event A in a very intuitive way. Assume my population size is 2000 and 100 of them have event A and the rest don't. I somehow came up with a cutting point for the age, e.g. less that 40 years old and greater than 40 years old. Here is the distribution of the population:

                  Have event A       don't have event A
less that 40              20                   100
greater than 40           80                   1800

To show the impact of age on event, I do the following : p(have event A| age less than 40) / p(have event A/ age greater than 40) = (20/120) / (80/1880)

However, I'd like to find something like a p-value for this calculation. Howe can I do that?


If you are okay with asymptotic variance using log transformation and delta method, then $$\text{SE}(\log(RR)) = \sqrt{\frac1{20}+\frac1{80}-\frac1{120}-\frac1{1880}}\approx 0.2316$$

With the observed $\log(RR)\approx 0.5929$ and assuming you want to test if $RR=1$ which is equivalent to $\log(RR)=0$, the $z$ statistic is about $2.5600$. So two-tailed p-value is about 0.0105.

In case you are interested, here is a link that shows derivation for the variance: Why doesn't standard error for ratios have log in it?

Exact confidence interval is also available, but not as easily obtained: How to calculate the "exact confidence interval" for relative risk?

  • $\begingroup$ is the SE(log(RR)) a confidence factor like p-value? $\endgroup$ – HHH Aug 17 '17 at 0:15
  • $\begingroup$ In a sense, yes. Standard error basically measures the spread/certainty of your estimate. Higher SE means you're less confidence in your estimate (wider confidence interval), meaning it is more likely to contain the value of your null hypothesis, leading to higher p-value. $\endgroup$ – Jirapat Samranvedhya Aug 17 '17 at 15:55
  • $\begingroup$ thanks, and may I know what values of SE is typically considered as high confidence (like p-value <0.05)? $\endgroup$ – HHH Aug 17 '17 at 16:02
  • $\begingroup$ Suppose the observed statistic is $z$, null hypothesis value is $\mu$, and standard error is $se$. Assuming normal distribution, the p-value for two-tailed test is $$P(|Z|>\frac{z-\mu}{se})$$ where $Z\sim N(0,1)$. That is, you also need to take into account of observed statistic and the null hypothesis. $\endgroup$ – Jirapat Samranvedhya Aug 17 '17 at 16:05
  • $\begingroup$ for the calculation you have above with SE=0.23, is it considered a high confidence or low? $\endgroup$ – HHH Aug 17 '17 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.