This is about a joint project with a subject matter expert, and I'm the statistician. I will not explain the background and simplify the statistics problem as the background is rather complicated. The model that we actually fit is a negative binomial mixed effects GLM, but the problem can be explained using a Poisson regression.
Imagine we have a response variable $Y$ which is a count, modelled as explained by some variables $X_1,\ldots,X_p$. There's also a variable $P$ that is a "population size" in some connection to $Y$, so that originally it seemed to be a good idea to my expert to model $Y/P$, i.e., "$Y$ by population member" rather than $Y$. I should add that from the subject matter information/meaning of the variables the situation isn't crystal clear, I mean, it's somewhat plausible but not totally obvious that $Y$ should be standardised by $P$. Complication is added by the fact that some of the X-variables are also standardised by P whereas others aren't. Again there are reasons for this, but they are also "somewhat plausible but not totally obvious".
The way to model $Y/P$ using a Poisson regression with log link is to include an offset in the model, i.e., a term $\log(P)$ with a fixed regression coefficient 1. I call this model M1. Out of curiosity we also had a look at a model M2 in which there is no offset but $\log(P)$ is included as another predictor variable. Fitting model M2, the $\log(P)$ variable has an estimated coefficient that is close to zero and (not borderline) insignificant. (This and what I write later doesn't change substantially if $\log(P)$ is replaced by $P$.)
Among the X-variables there are a few that are correlated with $\log(P)$ (not extremely strongly; correlations go up to 0.4). This explains, in my view, the fact that the estimated coefficients and their p-values in M1 are quite different from M2, to the extent that two predictors have the opposite sign but are strongly significant in both models, and another one seems very strong in M1 but is insignificant in M2.
Now my question is whether the insignificance and "near zero" status of $\log(P)$ in M2 is actually a strong reason to argue that M1 should not be used (and estimators and p-values should be interpreted from M2 regardless of whether they are so different in M1). If indeed the "true" coefficient of $\log(P)$ in M2 were zero, this would mean that (given the other variables) $Y$ would not be bothered about $P$ and therefore it seems to make sense to model $Y$ as in M2 rather than $Y/P$ as done in M1. Also of course an estimated coefficient for $\log(P)$, and be it zero, will fit the data better than fixing this coefficient to one, as the offset does (AIC and the like prefer M2). On the other hand, one could suspect that standardisation of $Y$ by $P$ could still be correct (given "plausible" subject matter arguments), only that in M2 its impact is "taken over" by other variables, and interpreting the resulting coefficients of the other variables in M2 may not represent the situation correctly.
The purpose here is the analysis of a historical situation, interpreting variable impact and importance. Prediction is not of relevance as an aim of this analysis.
Any insight?
