Assume there are a total of 1000000 contracts. Assume there are 120 bad contracts in total.

Assume that company 1 issues 500000/1000000 contracts and 70 are deemed bad. Assume that company 2 issues 100000/1000000 contracts and 50 are deemed bad

Assume there are lots of companies.

Clearly from the example above, company 1 is better than company 2, it issued 5 times as many contracts but only 20 more contracts are bad.

How do I go about ranking the companies based on who's most reliable at issuing contracts?

  • 1
    $\begingroup$ You can rank on number of bad contracts or on proportion of bad contracts. Statistical advice won't contradict what will be evident: you get to decide what is interesting or useful, but there is a warning that the interpretation of analyses will be different. $\endgroup$ – Nick Cox Nov 24 '15 at 12:37

You can simply calculate proportion of bad contracts,

$$ P = {n_b \over n}, $$

so that $P_1 = 1.4 \cdot 10^{-5}$ and $P_2 = 5 \cdot 10^{-4}$, but it's better to add a correction for total number of contracts issued. One way to correct is to add "dummy" bad contracts,

$$ P' = {n_b +10 \over n}. $$

Companies with a large number of contracts see their modified percentage alter very little from their real percentage ($P_1' = 1.6 \cdot 10^{-5}$ and $P_2' = 6 \cdot 10^{-4}$), but companies with relatively few contracts will see their modified percentage move considerably toward high values ($n_b =3$,$n= 30000$, so $P_3=1 \cdot 10^{-4}$, and $P_3'= 4.3 \cdot 10^{-4}$).

This is known as "Bayesian averaging". In effect, the companies with many contracts will rank higher than those companies with the same percentage but fewer contracts.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.