I recently started my PhD and I am currently working on a project about finding statistical reporting errors. Our work is similar to Nuijten et al. (2016) only for economics.
So, I have a database that consists of statistical tests (like a t-test) gathered from several journals. Additionally, by an easy calculation, I can find if this test might contain an statistical reporting error or not. Since I have several metadata about a test, like the year, the amount of authors, the amount of tests in the article, I want to do a regression that helps me explaining why such errors might occur.
Actually, I have not seen the data yet. That is why I want to do a pre-analysis plan. On of my hypotheses is that the amount of reporting errors gets lower if authors publish in a journal with open data and open code policy.
What would be a good model to check for this hypothesis? I thought about a poisson or a negative binomial model, since the dependent variable should be the amount of statistical tests with a reporting error that are prevalent an article. As exposure variable I could use the amount of tests in a table (the more tests, the more likely to make a mistake). As can be seen in a previous study around 50 % of the articles do not contain an error at all, while for a few articles 26 % of their respectives tests contain errors.
Do you have another idea or would you confirm this model choice? Some other authors do this on a test level by using a logistic regression if a certain tests contain an error or not. Although this seems reasonable, I think on an article level I have more chances for inferences.
Thank you a lot in advance!