I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal sentence outcome (prison vs. no prison) and the level 2 units are the judges handling the cases. I'm trying to quantify the effects of experience. For example, I want to know what happens when a judge handles more murder cases - does he become more lenient with the subsequent murder offenders or more punitive?
I see two ways to operationalize experience. 1. include a running count of the number of xyz type cases the judge has handled. 2. calculate a running proportion of the number of each judge's cases that are of each type.
In both cases, a cross level interaction term between the crime type variable and count/proportion variable is necessary.
There are 9 different crime categories, so I include 8 dummy variables. Each crime type dummy has a corresponding count (or proportion) variable that goes along with it. However, one of the 8 dummies is a catchall category - 'other' - so I didn't include a count for that because it doesn't really make sense.
The problem I'm having is that collinearity diagnostics suggest that the 7 count/proportion variables are problematic. The proportion variables look ok on some diagnostics and bad on others while the count variables look others and bad on some. I don't know why the diagnostics disagree and which ones I should trust. In models with either set of variables (count or proportion) the variables are generally statistically significant and the standard errors are not large but that's probably attributable to the large number of cases.
Here's how the diagnostics shake out.
Very high bivariate correlations of .7, .8, and even .9. The condition index calculated from uncentered, unstandardized variables with a constant (Stata's 'coldiag2) is 24.5 - which is high but acceptable. The VIFs on the other hand, calcuated using Stata's 'collin', are impossibly high - 18, 11, 17, 10, and a few 'low' ones under 5
Proportions: The correlations are all modest at about .2 or less, although one is as high as .5. The VIFs are also great, the mean VIF is just 1.44. But the condition number is even worse than the count variables - 32.5.
So the diagnostics disagree. I know the correlation matrix is not the best tool, but the VIFs and condition number wildly disagree and I'm not sure what to make of that.
Putting aside all that, I 'want' to use the count variables instead of the percentages.
Read on if you want to see why, but these issues are secondary to the collinearity issue outlined above.
The proportion variables take on extreme values (i.e. 100% or 1) for the first case and then regress toward the mean as the judge handles more cases. Thus, the change in the percentage variables really just reflects regression toward the mean. Every additional case is an additional 'unit' of experience and yet additional cases only change the percentage variables if they change the distribution of the judge's caseload. That's something that's hard to do when the judge has heard a thousand or more cases.
I think these limitations of the percentage variables can be at least partly addressed by the inclusion of a running count of the total number of cases the judge has handled but that's not a panacea and it will make the collinearity problem worse. On the other hand, the count variables are overdispersed as you would expect any count variable to be.
So, to summarize, the problems are twofold - difficulty operationalizing "experience" and disagreeing collinearity diagnostics. Perhaps there's a better way to operationalize experience than either counts or proportions that would not generate collinearity problems.
I would welcome any advice about alternative model specification OR correct interpretation of the collinearity diagnostics.