Logistic Regression Model inquiry with binary data

Question

The short version of my model is that I have ran a logistic regression (logisticmodel<-glm(FishObs~CGDD, family = binomial, data = dat1)) looking at when fish were observed (denoted by 0 or 1, 1 being observed) against thermal cues (CGDD). I ran this core model with three years of data (2015-2017) that I pooled together and did not account for year. I then used this core model and trained it on an additional year of data(2018) to look at misclass, sensitivity and specificity. 2015, 2017, and 2018 have 365 points of data. 2016 has 366 points of data for the leap year.

Where I am stuck is understanding some theory…

1. I was advised to pool data from (2015-2017) and train it against the 2018 data. But I am wondering if this model is inaccurate without considering year as a random factor. I am wondering under what conditions it makes sense or is appropriate to pool the data? I did run the model with year as a random factor, and the model worked but I was unable to determine the predicted probabilities. Perhaps limited by my R knowledge.

2. When using 2015-2017 data and training it against 2018, is there logic to this decision? My concern would be had I of ran 2016-2018 against 2015 for example, the outcome of the core model would be different.

Answer 1

I am wondering under what conditions it makes sense or is appropriate to pool the data?

The model without year as a factor looks like

$$ \operatorname{logit}(p) = \beta_0 + \beta_1 {CGDD} + \beta_2{t} $$

Where $t$ is a year factor. If $\beta_2$ is small enough to be negligible over the three years or so you choose to pool the data then it makes sense to pool.

We can actually test this assumption statistically. Fit two models (one with year and one without) and perform a likelihood ratio test. If you treat year as a continuous variable (and you should) this will be equivalent to the Wald test reported in the summary of the model.

When using 2015-2017 data and training it against 2018, is there logic to this decision?

This is something for you to convince us of. It makes most sense to forecast on unseen data from the future (I disagree that accuracy, sensitivity, and specificity are the best measures). Clearly, if the model is going to be used, it will be used to predict the future from the past (not the past from the future). But I would encourage you to justify your approach rather than asking the reader to infer proper justification.