Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal sentence outcome (prison vs. no prison) and the level 2 units are the judges handling the cases. I'm trying to quantify the effects of experience. For example, I want to know what happens when a judge handles more murder cases - does he become more lenient with the subsequent murder offenders or more punitive?

I see two ways to operationalize experience. 1. include a running count of the number of xyz type cases the judge has handled. 2. calculate a running proportion of the number of each judge's cases that are of each type.

In both cases, a cross level interaction term between the crime type variable and count/proportion variable is necessary.

There are 9 different crime categories, so I include 8 dummy variables. Each crime type dummy has a corresponding count (or proportion) variable that goes along with it. However, one of the 8 dummies is a catchall category - 'other' - so I didn't include a count for that because it doesn't really make sense.

The problem I'm having is that collinearity diagnostics suggest that the 7 count/proportion variables are problematic. The proportion variables look ok on some diagnostics and bad on others while the count variables look others and bad on some. I don't know why the diagnostics disagree and which ones I should trust. In models with either set of variables (count or proportion) the variables are generally statistically significant and the standard errors are not large but that's probably attributable to the large number of cases.

Here's how the diagnostics shake out. Counts:
Very high bivariate correlations of .7, .8, and even .9. The condition index calculated from uncentered, unstandardized variables with a constant (Stata's 'coldiag2) is 24.5 - which is high but acceptable. The VIFs on the other hand, calcuated using Stata's 'collin', are impossibly high - 18, 11, 17, 10, and a few 'low' ones under 5

Proportions: The correlations are all modest at about .2 or less, although one is as high as .5. The VIFs are also great, the mean VIF is just 1.44. But the condition number is even worse than the count variables - 32.5.

So the diagnostics disagree. I know the correlation matrix is not the best tool, but the VIFs and condition number wildly disagree and I'm not sure what to make of that.


Putting aside all that, I 'want' to use the count variables instead of the percentages.

Read on if you want to see why, but these issues are secondary to the collinearity issue outlined above.

The proportion variables take on extreme values (i.e. 100% or 1) for the first case and then regress toward the mean as the judge handles more cases. Thus, the change in the percentage variables really just reflects regression toward the mean. Every additional case is an additional 'unit' of experience and yet additional cases only change the percentage variables if they change the distribution of the judge's caseload. That's something that's hard to do when the judge has heard a thousand or more cases.

I think these limitations of the percentage variables can be at least partly addressed by the inclusion of a running count of the total number of cases the judge has handled but that's not a panacea and it will make the collinearity problem worse. On the other hand, the count variables are overdispersed as you would expect any count variable to be.


So, to summarize, the problems are twofold - difficulty operationalizing "experience" and disagreeing collinearity diagnostics. Perhaps there's a better way to operationalize experience than either counts or proportions that would not generate collinearity problems.

I would welcome any advice about alternative model specification OR correct interpretation of the collinearity diagnostics.

share|improve this question
2  
I suspect you will get loads of TL;DR reactions to this question, which doesn't bode well for the prospect of getting great answers. Do you think you could highlight the key information and remove the less essential parts of this question? On my first reading, the really important part seems to be the sudden switch from counts to proportions. Are you asking why a regression based on proportions can have different properties than a regression based on counts? –  whuber Jun 13 '12 at 15:10
1  
There are 2 basic issues: 1. I'm not sure if proportions or counts or something else is the best way to capture 'experience' 2. Neither set of variables (proportions or counts) does well on ALL the standard collinearity diagnostics.. BUT BOTH do well on different diagnostics. I don't know why the diagnostics disagree and which ones I should trust. I'll edit the question to better highlight and organize these issues. –  Will Jun 13 '12 at 16:42
    
Have you considered using proportion data for the types of cases along with a count column for general experience (i.e. total number of cases heard), or have I misread your question? –  Jonathan Thiele Jun 13 '12 at 17:38
    
Jonathan, I think if I do go with proportions I will have to include a running count of the total cases handled. That should control for the tendency of percentage variables to decrease over time (i.e. from 100% toward the mean). I'm leaning toward that approach as I think about it more, which is unfortunate because the models and post estimated marginal effects and charts and everything else I've done thus far will have to be changed. BTW, the model does include variables for judge's age and time on bench - so those somewhat capture 'experience' as well. –  Will Jun 13 '12 at 18:50
    
Jonathan, to follow up - if I do as you suggest, what do I make of the collinearity diagnostics? The bivariate correlations are fine and the VIFs are great... but the condition index just among the proportions (not the full set of predictors) is over 30 which, I think, is very bad. –  Will Jun 13 '12 at 19:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.