Can logistic regression estimates suffering from subsample abuse be salvaged?

Question

Suppose we have some logistic regression modelling problem; $f(X) = Y$, where $Y$ is binary and $X$ is a vector of normally distributed variables.

In industry it is sometimes the case that practitioners will delete rows of data where 'nothing happens' (according to an algorithmic criteria), often to reduce the dimensions of the regression (data can get quite big) or because of the perception that this 'nothing happening' is not what we want to model - we want the model only to capture when interesting events occur.

Needless to say this can bias out of sample probability estimates (i.e., given some new $X$, our estimate of $P(Y=1)$ is biased). A quick R demonstration will show this, where the true data generating process has $Y=1$ half the time and $Y=0$ the other half but our subsampling approach has this ratio to $\frac19$:

predictions <- rep(NA,1000)

y <- c(rep(1,5000),rep(0,45000))
x <- rnorm(50000)
#y <- c(rep(1,25000),rep(0,25000))
#x <- rnorm(50000)

fit = glm(y~x,family=binomial(logit))

for(i in 1:1000) {
    newX = data.frame(x=rnorm(1))
    predictions[i]=predict(fit,newX,type="response")
}

    mean(predictions)
plot(predictions)

Question

What I would like to ask is whether our fit, $\hat{f}$, can be salvaged in the following way. For every new $X$ that we observe, we only estimate $\hat{P}(Y-1)$ if and only if it would have passed through our filter had this new $X$ been part of our in sample dataset. Intuitively I want to say yes because in-sample we have the data generating process that we artificially created, $f_\text{intrusion}(X) = Y_\text{intrusion}$, and this is the same DGP that we are drawing from out of sample to get our probability estimates.

Answer 1

In logistic regression, oversampling events does not affect non-intercept coefficients - they remain unbiased. Only the intercept $\beta_0$ is affected, and this is what drives the incorrect probability predictions. So if you only cared about the coefficients for explanatory variables and not the actual predicted probabilities, you don't need to do anything, but that's not the case here.

See King and Zheng (2001) [equation 7] for the direct correction to $\beta_0$

$$ \hat{\beta_0} - ln[({1-\tau \over \tau})({\hat{y} \over 1-\hat{y}})] $$

$\hat{\beta_0}$ being the incorrectly estimated intercept
$\tau$ being the proportion in the actual population
$\hat{y}$ being the proportion in the sample population

So for your simple example, you should have something like ${\beta_0} = -2.2 - ln[({0.5 \over 0.5})({0.1 \over 0.9})] = -2.2 + 2.2 = 0 $.