Poisson regression intercept downward bias when true intercepts are small

Question

When fitting a Poisson regression on data with low expected values, the intercept term has a small bias even when the model is perfectly specified. Below, I simulated data just using $y \sim rPois(exp(\beta_0))$ and then fit the data using the glm model $log(E[y]) \sim \beta_0$. On average, the estimates are slightly biased downwards. The bias is small, but I would like to understand why this happens.

I could understand why this would happen if $\beta_0$ was a large negative number and the data were mostly zeros, but the data from the $\beta_0$ values I chose is always mostly non-zero. Why would this happen?

# function to run the simulation for one set of beta values
    run_sim <- function(b0, n = 50, R = 10000){
      
      # simulate y values and then estimate
      b0_estimates <- sapply(1:R, function(i){
        y = rpois(n, exp(b0))
        tmp = data.frame(y = rpois(n, exp(b0)))
        mod_col <- glm('y ~ 1', data = tmp, family=poisson)
        b0_hat <- mod_col$coefficients[1]
        return(b0_hat)
      })
      
      # get the bias
      mean_bias = mean(b0_estimates) - b0
      
      return(mean_bias)
    }
    
    # simulate for beta0 values ranging from 1 to 10
    b0_vec = 1:10
    bias_vec = sapply(b0_vec, function(b0){
      run_sim(b0, R = 10000)
    })
    
    # plot the results
    plot(b0_vec, bias_vec, xlab = 'true b0', ylab = 'b0 bias')

Answer 1

The score function is exactly unbiased $$E_{\beta_0}[\sum_i x_i(y_i-\mu_i)]=0$$ In your case that simplifies to $$E_{\beta_0}[\sum y_i-\exp\beta_0]=0$$ The parameter estimate is a non-linear function of the score, so that tells us it won't be exactly unbiased.

Can we work out the direction of the bias? Well, the mean of $Y$ is $\exp \beta_0$, so $\beta_0=\log EY$ and $\hat\beta=\log \bar Y$. The logarithm function is concave, and $E[\bar Y]=E[Y]=\exp\beta_0$ so we can use Jensen's inequality to see that the bias is downward. (Or draw a picture, like the one here only the other way up)