Calculate a 95% Confidence Interval of a population proportion in R

Question

I have this data:

 structure(list(age = c(62.84998, 60.33899, 52.74698, 42.38498
 ), death = c(0, 1, 1, 1), sex = c("male", "female", "female", 
 "female"), hospdead = c(0, 1, 0, 0), slos = c(5, 4, 17, 3), d.time = c(2029, 
 4, 47, 133), dzgroup = c("Lung Cancer", "Cirrhosis", "Cirrhosis", 
 "Lung Cancer"), dzclass = c("Cancer", "COPD/CHF/Cirrhosis", "COPD/CHF/Cirrhosis", 
 "Cancer"), num.co = c(0, 2, 2, 2), edu = c(11, 12, 12, 11), income = c("$11-$25k", 
 "$11-$25k", "under $11k", "under $11k"), scoma = c(0, 44, 0, 
 0), charges = c(9715, 34496, 41094, 3075), totcst = c(NA_real_, 
 NA_real_, NA_real_, NA_real_), totmcst = c(NA_real_, NA_real_, 
 NA_real_, NA_real_), avtisst = c(7, 29, 13, 7), race = c("other", 
 "white", "white", "white"), sps = c(33.8984375, 52.6953125, 20.5, 
 20.0976562), aps = c(20, 74, 45, 19), surv2m = c(0.262939453, 
 0.0009999275, 0.790893555, 0.698974609), surv6m = c(0.0369949341, 
 0, 0.664916992, 0.411987305), hday = c(1, 3, 4, 1), diabetes = c(0, 
 0, 0, 0), dementia = c(0, 0, 0, 0), ca = c("metastatic", "no", 
 "no", "metastatic"), prg2m = c(0.5, 0, 0.75, 0.899999619), prg6m = c(0.25, 
 0, 0.5, 0.5), dnr = c("no dnr", NA, "no dnr", "no dnr"), dnrday = c(5, 
 NA, 17, 3), meanbp = c(97, 43, 70, 75), wblc = c(6, 17.0976562, 
 8.5, 9.09960938), hrt = c(69, 112, 88, 88), resp = c(22, 34, 
 28, 32), temp = c(36, 34.59375, 37.39844, 35), pafi = c(388, 
 98, 231.65625, NA), alb = c(1.7998047, NA, NA, NA), bili = c(0.19998169, 
 NA, 2.19970703, NA), crea = c(1.19995117, 5.5, 2, 0.79992676), 
     sod = c(141, 132, 134, 139), ph = c(7.459961, 7.25, 7.459961, 
     NA), glucose = c(NA_real_, NA_real_, NA_real_, NA_real_), 
     bun = c(NA_real_, NA_real_, NA_real_, NA_real_), urine = c(NA_real_, 
     NA_real_, NA_real_, NA_real_), adlp = c(7, NA, 1, 0), adls = c(7, 
     1, 0, 0), sfdm2 = c(NA, "<2 mo. follow-up", "<2 mo. follow-up", 
     "no(M2 and SIP pres)"), adlsc = c(7, 1, 0, 0)), row.names = c(NA, 
 4L), class = "data.frame")

I have also calculated the estimated population proportion of patients who had lung cancer as the primary disease group below.

SB_xlsx_mean = round(100 * mean(SB_xlsx$dzgroup == "Lung Cancer", na.rm = TRUE), 2)

SB_xlsx_mean
## [1] 9.97

The population proportion with the main disease type of lung cancer was 0.0997 or 9.97%.

However, now need to calculate the 95% CI of the population proportion of patients who had lung cancer as the primary disease group. I've gotten 95% CIs before with t-tests, but I don't think that is really applicable here and I'm not sure how else to start. Could someone assist?

Answer 1

I could not follow your R code. However, there is no single "correct" answer.

Suppose you have $x = 37$ "Successes" in $n = 150$ trials. There are many styles of 95% confidence intervals in common use. I will give several examples. Opinions vary about which is best and in which applications.

The Wald interval uses $\hat p = 37/150 = 0.2467$ and the corresponding 95% CI $(0.095,0.398)$ is of the form $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}.$ It is most useful for large $n.$ (I'd say $n > 500).$

p.hat = 37/150
CI.w = p.hat + qnorm(c(.025,.975))*sqrt(p.hat*(1-p.hat)/n)
 CI.w
[1] 0.09492108 0.39841225

The Agresti-Cooil 95% CI $( 0.185, 0.322)$ uses a slightly different estimate of the population proportion and has better properties for smaller $n.$

p.est = (37+2)/(150+4)
CI.ac = p.est + qnorm(c(.025,.975))*sqrt(p.est*(1-p.est)/154)
CI.ac
[1] 0.1845639 0.3219296

The Jeffreys 95% CI $(0.183, 0.320),$ based on a Bayesian argument with a non-informative prior uses a beta distribution.

qbeta(c(.025, .975), 37.5, 150.5 - 37)
[1] 0.1829657 0.3200351

The Clopper-Pearson 95% CI $(0.180, 0.324)$ is implemented in the binom.test procedure in R:

binom.test(37, 150)$conf.int
[1] 0.1800065 0.3236100
attr(,"conf.level")
[1] 0.95

When the R procedure prop.test is used with a single group, it provides a confidence interval, here $(0.182, 0.325),$ for the group population proportion:

prop.test(37, 150)$conf.int
[1] 0.1816347 0.3249513
attr(,"conf.level")
[1] 0.95

I include this interval on my list because I just discovered it today and I have not yet been able to decipher the documentation for prop.test to know the name of this interval---if there is one. I hope one of my colleagues on this site will know. (It seems close to Clopper-Pearson.)

Notes: For more, see the links under 'Related' in the margin and this along with its links. Also, you can google 'binomial confidence interval' to see several other kinds of frequently used CI's that may be relevant for your purposes.