5
$\begingroup$

The UN has a convention against corruption (UNCAC) that has been signed by some 140 countries and ratified by most (link).

Transparency International publishes an annual report on corruption. They have data for most countries for the period 2000-2010, each country is given a score between zero and ten where ten is best (lowest level of corruption). There is usually a slight trend in the corruption score, indicating that there is some inertia in the level of corruption (link).

I want to test whether ratifying the convention has any positive effect on the level of corruption in the country.

My initial idea is to calculate the mean level of corruption 3 years before and after signing the treaty and using a students paired t-test to see if latter mean is significantly larger than the former.

Another approach, that I would know less about, is to use some kind of event model where dummy variables are used for the years after the convention is ratified.

What are your thoughts on this? Would the first approach work, is there anything I have overlooked? All feedback is appreciated.

Notes

I should note that this is a minor part of a course term paper. We weren't supposed to do anything quantitative, just a write-up on a topic of our choice. However I want to take a quick look at the statistics since data is readily available and I prefer to look at numbers if possible. Hence, I don't need the ideal PhD-level statistical model, just something quick and simple.

$\endgroup$
5
$\begingroup$

It's probably not a good idea without digging deeper into the source surveys. TI themselves note that changes in a country's index can come from either a change in corruption or just a change in methodology of the sources that they use. The sources themselves change over time as well, so comparing the index from year to year is actually fairly complicated to do properly.

See the references at http://en.wikipedia.org/wiki/Corruption_Perceptions_Index

$\endgroup$
  • $\begingroup$ I'll look into it, thanks! There are a few other indices and I think some of them have been used in analysis over time, so there may be other datasets that don't exhibit the problem you've highlighted. $\endgroup$ – sandstrom Mar 12 '11 at 13:31
  • $\begingroup$ Oh plenty of people have used the CPI to make comparisons over time. But it's a terrible idea without accounting for 1) changes in the surveys used to make the composite index and 2) changes in the methodology of those surveys. Like you say I would imagine there are indices out there which try to make their measurements comparable over time, but TI doesn't. $\endgroup$ – JMS Mar 12 '11 at 18:33
4
$\begingroup$

One problem with your pre-post paired t-test idea is that it chould give a small p-value if there's a general upward (or downward) trend in the corruption score over time regardless of ratification. You need some sort of comparison group in which the ratification status did not change between the two time points.

In principle, one approach to forming a comparison group could be to find a matched 'control' country that didn't change its ratification status during the same time period for each country (of a sample of those) that did. The 'control' country should have similar baseline corruption score, and ideally be as similar as possible in other respects that might affect the rate of change of corruption over time (that's what I mean by matching). I haven't looked at the data so I've no idea if this is feasible. You'd want to consider removing the country that changed status from your analysis if you can't find a 'control' country that matches closely enough (but what's 'close enough'?).

In any case it certainly wouldn't be simple, but drawing valid and defensible conclusions about causal effects from observational data seldom is. I agree with @JMS that it's probably not worth attempting in your situation.

$\endgroup$
  • $\begingroup$ Perhaps it would be a better idea use the slope (before and after), to accommodate the fact that there may be a trend? $\endgroup$ – sandstrom Mar 12 '11 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.