I'm investigating whether a federal district court judge's ABA rating (rating given to the judge by the American Bar Association when he/she is nominated) significantly correlates to the rate that his/her opinions are reversed on appeal. (I'm in the USA.) There are 645 judges total. My data looks like so:
|Judge4||Female||Bush II||Republican||Well Qualified||129||97||32||0.24806202|
From the very helpful Cross Validated community, I believe I should be doing logistic regression using the glm() function. I am using the following model:
model_X <- glm(cbind(aff_tot, rev_tot) ~ [explanatory variable(s)], data = dct_data, family = binomial)
cbind(aff_tot, rev_tot) as my dependent variable instead of a simple
rev_rate because I want to account for the fact that judges have had different numbers of cases appealed.)
I'm sure this is just because I'm new to R/new to stats, but no matter what explanatory variable(s) I used, I almost always get really small p-values (examples below), even when it obviously doesn't make sense to have such small p-values. Am I setting my dataset up incorrectly? Am I using the
glm() function incorrectly? I've asked several stats friends, and they couldn't figure it out.
[If it's helpful, I'm here are some examples...]
Example 1: Gender
All judges are either Female or Male. The average reversal rate for Females is 12.2%, and the average for Males is 13.5%. But when I use the below model, I get a p-value of 2x10-16 for the coefficient on Females and 2.5x10-8 for Males.
model_gender <- glm(cbind(aff_tot, rev_tot) ~ gender, data = dct_data, family = binomial)
Example 2: President Party
All judges were nominated either by a Democrat or a Republican. The average reversal rate for Democrat-appointed judges is 13.5%, and the average for Republican-appointed judges is 13.0%. But when I use the below model, I get a p-value of 2x10-16 for the coefficient on Democrats and 0.0203 for Republicans.
model_president_party <- glm(cbind(aff_tot, rev_tot) ~ president_party, data = dct_data, family = binomial)
With 645 judges, my instinct is that it's impossible that a ~1% difference in reversal rates for Female vs. Male should yield such a low p-value. (Ditto for the 0.5% reversal rate difference for Democrats vs. Republicans.) But whenever I look for a correlation between reversal rate and any of the explanatory variables, or whenever I combine multiple variables in the same model, I almost always get really low p-values for each explanatory variable. Again, I'm sure I'm making a dumb mistake, but I'd really appreciate it if someone could show me what I'm doing wrong.