I am monitoring variability in sentencing outcomes (custodial sentence length) amongst matched groups. The aim is to determine whether sentence variability went up or down after a policy was implemented.
I have about 40 groups of sentences, which are matched on the characteristics of the case. For example, group 1 comprises sentences for people who committed ABH, who were under the influence of drugs, and used a weapon. Group 2 comprises people who committed ABH, who were remorseful, and had no aggravating factors in the case.
For each group, I can further separate sentences outcomes into two groups: those sentenced before a policy was implemented, and those sentenced after the policy was implemented. I can then compare sentenced outcomes in the 'before' and 'after' groups to see whether variability seems to have declined.
I can test each group individually to see whether variability of sentence outcome went up or down, using an F test. However, the results of these tests vary, and I realise that I should really be using a joint test in any case.
I'm also not getting enough power on a group-by-group basis: for most tests, we cannot reject H0, that variability did not change. This is despite the fact that variability has declined for the vast majority of groups, which suggests that if I somehow combine results from all groups, it should be possible to find a statistically significant decline in variance overall.
Could anybody suggest and appropriate test or how I might proceed? I am very grateful for any help.
A few clarifications:
- I would be happy to assume that amongst a matched group (log) sentence outcomes are drawn from a normal distribution. This assumption will not be perfect, but is probably adequate.
- The size of matched groups differs. For instance, the group 1 'before' implementation group may have 100 sentences, and the 'group 1 after implementation' group may have 75 sentences. The same figures for group 2 may be 66 and 123.