1
$\begingroup$

I have a dataset reporting a number of incidents for a given population, such as: 

Country Nb incidents    Population  Rate
A             1               30    3.33%
B            71            15000    0.47%
C             4             2000    0.20%
D             1              600    0.17%
E            19            12000    0.16%
F             3                3     100%

My boss asked me to ignore countries having only one incident. Country A is incorrectly flagged as a problem, so we should not take it into account at all and concentrate on country B instead.

But I'd like to implement a more robust and efficient solution. My problem is that country A has the highest Rate of incidents, but it is obviously not significant since the tested population is very small. Country D has also only one incident but the tested population is large enough.

As an extreme example, I have added a country F where the rate is 5000%. Here, we have to consider the result because a lot of incidents for a small population would mean we have a huge problem ;)

I thought chi-squared would be OK, but I have no idea how I could use it on this data set.

Improve this question
$\endgroup$
11
  • 2
    $\begingroup$ What does it mean to "rip" a country? What are you ultimately trying to do (ie, what will you do after ripping)? $\endgroup$
    – gung - Reinstate Monica
    Feb 28, 2018 at 15:58
  • $\begingroup$ I have edited my post, what I meant was that we should ignore this entry completely. Ultimately, we want to priorize actions : the country with the highest rate should be considered first to prevent further incidents from happening $\endgroup$
    – Mike
    Feb 28, 2018 at 16:05
  • 2
    $\begingroup$ So the idea here is your firm wants to focus its efforts on the country w/ the biggest problems, as measured by the observed rate of incidents, is that right? Are you just trying to identify the worst, or do you need to rank all of them, or do you want the best estimate of their true rates? $\endgroup$
    – gung - Reinstate Monica
    Feb 28, 2018 at 16:18
  • $\begingroup$ The biggest absolute problem is wherever there are most incidents. The biggest relative problem is wherever the rate is highest. If data are known independently to be wrong, ignore them. Sorry, but I am not clear what else is there is to say. (Evidently your data are just fake; we understand, but there is no analysis to do there.) $\endgroup$
    – Nick Cox
    Feb 28, 2018 at 16:22
  • $\begingroup$ To start with, it would be OK to correctly identify the worst without wrongly flagging country A as the worst (since it is an obvious outlier). I am looking for a more general method than just "deciding" that 1 incident for 30 population is not OK (why not 2 for 30 e.g.). So, yes, the idea is to focus on the country with the biggest problems as measured by the rate of incidents $\endgroup$
    – Mike
    Feb 28, 2018 at 16:23

1 Answer 1

Reset to default
2
$\begingroup$

I would not use the chi-squared test for this. You don't need a test to see if the proportions differ, you believe they differ—that's why you are trying to determine the 'worst'. There can't be a worst if they are all equal. Instead, you want a ranking. To get a ranking, we need to estimate some underlying quantity on which the ranking is based. In your case, you need to estimate an underlying probability (I think, or maybe the rate) of having an incident.

The problem is that you have differing amounts of information on which to base your estimates. This is actually a pretty common problem in a ranking context. What you need is something that will adjust the observed proportion itself to account for the differences in information. I suggest a more or less Bayesian approach. First, I suspect the different types of incidents are correlated due to common issues like better or worse maintenance (or perhaps proximity to the sea). I would compile a matrix from all countries (in rows) and all incident types (in columns), with each cell being the observed proportion. Then, I would compute the correlation matrix by columns and run a principle components analysis. See how many PCs seem reasonable. My guess is that you can cluster your incident types into at most a few groups, possibly even only one. Within each group of incidents, average the observed proportion within each incident type, and then average over the averages. (If you have a complete, balanced dataset, this is the average over the matrix.) This becomes your baseline proportion for all incidents within that group. Then determine the population sizes, and compute their average likewise. This is how much information the baseline average is typically based on. (At one extreme, you would have a single baseline and population average for the entire dataset, at the other extreme, you would have a different baseline and population average for each incident type.) With these two pieces of information, you can adjust your observed proportions. Specifically, just do a weighted average of your observed proportion with your baseline proportion, with the observed population size and average population size as the weights. All estimated proportions will be moved at least somewhat towards the baseline average, but small populations will be moved more and large populations will be moved less. That's it. The worst country is the one with the worst adjusted probability of incidents.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for this answer, I have started an analysis using your advice, that seems to be working of a reduced dataset. $\endgroup$
    – Mike
    Mar 6, 2018 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.