First off I apologize, that I cannot share the code or details about the variables for this project. I am new to statistics and am working on a project using count data. I want to make sure I am going about this correctly so I would appreciate any feedback. I am trying to build a predictive model using count data. I have done the following:
1) Looked at the distribution of the dependent variable as well as the independent variables and most if not all follow the Negative Binomial Distribution. Given that the data follows this distribution and I am using count data I decided to use negative binomial regression. Additionally checked to see how the mean of the target variable compared to its variance and since it was not equal, I ruled out poisson regression.
2) In R I used the MASS package and specifically used the glm.nb function with the syntax of glm.nb(y ~., data = data). For the initial run I included all of the variables
3) For the remaining runs, I removed 1 variable at a time at a time (with the highest p-value) and re-ran the model until there were no p-values above 0.05. for each iteration I logged the variable that was removed and the AIC of that model.
At this point I am not sure what is the "correct" next step for this model.
My main questions are: 1) What are some of the next steps that I should take? - I was thinking of plotting the actual values to the predictive values of each model. - I was thinking about plotting the residual values for each predictive value to see how off the model was. - The only reason I was planning to do this was because I did this in the past for OLS, however, I wanted to confirm if this was applicable to Negative Binomial Regression as well.
2) What are some evaluation techniques I should use? I was thinking since I have 150K+ rows of data that I could use train/test split rather than using cross validation but would also be curious to hear any thoughts on this topic as well.