When there are many more controls than cases, can I take only part of the controls?

Question

I've tried to build a predictive logistic regression model. However, there are only 500 observations with disease (+) and over 60,000 observations with disease (-).

Can I take a random sample (e.g., 10,000) of 60,000 observations with disease (-) to build the predictive model?

Answer 1

Can I take a random sample (e.g., 10,000) of 60,000 observations with disease (-) to build the predictive model?

Let's try it out. The code below simulates data and fits two times the data one time without and one time with the reduction in data.

We see that the result is nearly the same. The reason that the reduction does not matter so much is because the estimate of the control group is very accurate already.

In the code we have computed the logistic model in two different ways.

• Using a glm model, which is the typical and straightforward approach
• Estimating the distribution of the two groups. This is not the typical approach (it requires a normal distribution of the two groups, control and cases), but it is comparable. (you can see the black line and the thick dotted red line coincide)

So what happens when you reduce the data is that your estimate of the control group becomes less accurate. However, the limiting factor is the estimate of the group with the cases. The cases group is smaller and the estimates are less accurate which dominate the overal estimation. So reducing the control group has little influence.

Of course, when you have all the data already, then there is no reason to reduce the groups. But when you still have to gather the data then it can make sense to safe resources by not measuring the entire control group.

Note, you do need to keep in mind the frequency of cases in the population of interest and not use the frequency of cases in the sample with a relatively higher frequency of cases. In the code below this is done by correcting the odds from the fit with the sample according to the frequency of the population. This is done in the line odds2 = odds*(reduce*nx+ny)/(nx+ny).

layout(matrix(1:2,2))

### generate two groups of data
set.seed(1)
nx = 6*10^4
ny = 500
y = rnorm(ny,2,1)
x = rnorm(nx,0,1)
z = c(x,y)
cat = c(rep(0,nx), rep(1,ny))

### plot
plot(z,cat)

### compute logistic model by approximating normal distributions
mu_x = mean(x)
mu_y = mean(y)
sig = ( (sum((y-mu_y)^2) + sum((x-mu_x)^2)) /(ny+nx-2))^0.5 

zs = seq(-4,6,0.1)
p_caty = (dnorm(zs,mu_y,sig)*ny)/(dnorm(zs,mu_x,sig)*nx+dnorm(zs,mu_y,sig)*ny)
lines(zs,p_caty, lty = 1)

### compute logistic model with glm
mod = glm(cat ~ z, family = binomial)
lines(zs, predict(mod, newdata = list(z=zs), type = "response"), col = 2, lty = 2, lwd = 2)

title("estimate with all data 60000 vs 500 cases", cex.main = 1)

### generate two groups of data
set.seed(1)
nx = 6*10^4
reduce = 1/6
ny = 500
y = rnorm(ny,2,1)
x = rnorm(nx*reduce,0,1)
z = c(x,y)
cat = c(rep(0,nx*reduce), rep(1,ny))

### plot
plot(z,cat)

### compute logistic model by approximating normal distributions
mu_x = mean(x)
mu_y = mean(y)
sig = ( (sum((y-mu_y)^2) + sum((x-mu_x)^2)) /(ny+nx*reduce-2))^0.5 

zs = seq(-4,6,0.1)
p_caty = (dnorm(zs,mu_y,sig)*ny)/(dnorm(zs,mu_x,sig)*nx+dnorm(zs,mu_y,sig)*ny)
lines(zs,p_caty, lty = 1)

### compute logistic model with glm
mod = glm(cat ~ z, family = binomial)
out = predict(mod, newdata = list(z=zs), type = "response")
odds = out/(1-out)
odds2 = odds*reduce
out2 = odds2/(odds2+1)
lines(zs, out2, col = 2, lty = 2, lwd = 2)

title("estimate with reduced data 10000 vs 500 cases", cex.main = 1)