1
$\begingroup$

I'm working on count data preparation which will be used in Poisson/GLM. Specifically for year 2002 both males and females rows are missing as no event has been recorded (0 counts). Therefore should I complete the data with missing combinations filled with 0 counts or I can leave it as it is? What would happen to Poisson model if I wouldn't include these 0's? Overestimation?

Originally:

ID Sex Year Events
 1  M  2000     10
 1  M  2001      8
 1  M  2002     12
 1  F  2000      6
 1  F  2001      4
 1  F  2002      9
 2  M  2000     11
 2  M  2001      9
 2  M  2002     14
 2  F  2000      7
 2  F  2001      5
 2  F  2002     11
 3  M  2000     11
 3  M  2001      9
 3  F  2000      7
 3  F  2001      5

Proposed:

ID Sex Year Events
 1  M  2000     10
 1  M  2001      8
 1  M  2002     12
 1  F  2000      6
 1  F  2001      4
 1  F  2002      9
 2  M  2000     11
 2  M  2001      9
 2  M  2002     14
 2  F  2000      7
 2  F  2001      5
 2  F  2002     11
 3  M  2000     11
 3  M  2001      9
 3  M  2002      0
 3  F  2000      7
 3  F  2001      5
 3  F  2002      0
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Are these true 0 counts, or are the values missing for some other reason? $\endgroup$
    – EdM
    Commented Aug 18, 2022 at 14:40
  • $\begingroup$ I'm sorry I haven't seen the notification that you responded me. My counts are true 0 counts, just no event happened for both males and females but it was possible (it is not structural 0). So I guess I'm ok to complete the observations and go with regular Poisson model? $\endgroup$
    – Tom
    Commented Sep 8, 2022 at 12:47

1 Answer 1

Reset to default
1
$\begingroup$

If the observations are true 0 values rather than missing, then they should be included in the model as 0 values. Otherwise you would bias your estimates upward.

Although a Poisson model is commonly used for counts, you might need to consider a zero-inflated Poisson, quasi-Poisson, or negative binomial model to deal with over-dispersion. Pay attention to that possibility as you proceed with your analysis.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Exactly, I am already using negative-binomial as mean-variance relation is above linear. However if 0's in my data are random should I go for zero-inflated models? I always thought that these models are for data with structural 0's? $\endgroup$
    – Tom
    Commented Sep 8, 2022 at 16:39
  • 1
    $\begingroup$ @Tom a zero-inflated model does assume that there is a systematic reason for having excess 0 values that you can model from your data. If the 0 values truly occur at random, then it would be better to focus instead on a negative binomial. I was struck by both 0 values coming in the same calendar year in your data excerpt, suggesting a possible systematic reason for excess 0 values. Trust your sense of the data, as informed by your understanding of the subject matter, but recognize that what seems to be "random" might not be and might be amenable to modeling. $\endgroup$
    – EdM
    Commented Sep 8, 2022 at 16:47
  • $\begingroup$ Based on a similar post, I'm wondering what the correct answer is here? (the questions seem aligned). stats.stackexchange.com/questions/553453/… $\endgroup$
    – LucaS
    Commented Jul 19, 2023 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.