# How can you classify the accuracy of a probability?

Say I do a study of people that like bananas in 2 different regions.

• Region 1: 8 out of 10 people tested like bananas.
• Region 2: 500 out of 1000 people tested like bananas.

So in region 1, 80% of people like bananas. In region 2, 50% of people like bananas. I have a dilemma if want to sell bananas in either region 1 or 2. The accuracy of the probability in region 1 will be worse than in region 2. How can I quantify this accuracy to make a decision, in which region should I sell bananas?

• I think you're confusing the concepts of accuracy and precision. Without any additional information, we don't know whether 50% or 80% is the more accurate estimate. We can say, however, based on the sample size that the precision of the 50% estimate is much higher and thus our variability is much lower. Feb 20, 2016 at 0:08

This sounds like this question could be rephrased as "Is the proportion of people who like bananas in Region 1 significantly different than in Region 2?"

Reframed that way, the statistical test would be for the z-test for the difference between two proportions.

Here is a link to an online calculator: http://epitools.ausvet.com.au/content.php?page=z-test-2

Note, however, that Region 1 (at least in your example), is undersampled, and the statistical tests may be inaccurate. Many sources recommend sampling until you have at least 10 people who meet each condition (in this example, 10 people who like and 10 who dislike bananas in each region).

• I like your reasoning. But why not recommend using a test that is suitable for the data (such as a Binomial test) rather than suggesting that more samples should be collected?
– whuber
Feb 22, 2016 at 14:25

In this particular case, accuracy/precision and other sensible considerations don't matter. If the information that you gave in the question is all you know, then you have to sell bananas in region 1. Period.

However, if the price of bananas in region 2 was higher than in region 1, it would have been a different story. The reason is that the price asymmetry would bring the cost of error asymmetry.

What if in reality the region 2 liked bananas more than region 1? In this case although given the data it seems that region 1 has higher propensity to buy bananas, depending on the price differential the cost of error could be too high given the variances of your estimates.

UPDATE:

@whuber brought up an interesting topic: risk aversion. If you're averse to risks, then you may consider expected utility theory, it's studied in microeconomics and game theory. The trouble is that there's not enough data to set up the expected utility function.

For instance, in finance the simple examples are usually like follows. You have two stocks. Stock 1 returns 80% annually with standard deviation 60%, Stock 2 returns 50% annually with standard deviation 5%. Which stock to choose?

You need a utility function, such as $U(\mu,\sigma)=\mu-\sigma^2$. In this case $U_1=44%$ and $U_2=0.4975$., so you have to go with the Stock 2 despite its average return us lower, because it's risk is "too high".

• I think anybody who is aware of and concerned about utility and risk aversion in decision making would strongly disagree with your opening paragraph. Region 1 is a riskier prospect because the information about it is less certain. That issue should not dismissed out of hand. Don't you suppose this is the very reason the question was posed?
– whuber
Mar 23, 2016 at 18:11
• @whuber, why is Region 1 riskier? Is it because of your prior knowledge that generally people dislike bananas? Based on the information in the question there's nothing to suggest that Region 1 should have lower than 50% people liking bananas. Mar 23, 2016 at 18:25
• Consider a Bayesian analysis (because it's easier to explain) and suppose you adopt a weak conjugate prior of a $B(1,1)$ distribution on the proportion $p$ of people who will buy bananas. To oversimplify, suppose further that (1) you cannot tolerate a loss and (2) losses are likely when $p\le 0.45$. Then in region (1) the posterior chance of a loss is $3.6\%$ while in region (2) the posterior chance is only $0.08\%$. This reflects risk. A more nuanced analysis would use a more realistic loss function, incorporate utility, and assess the sensitivity to the assumed prior distribution.
– whuber
Mar 23, 2016 at 18:33
• @whuber, but you brought in a lot of stuff which was not in OP's question. He didn't indicate any loss aversion. He might be risk seeking or neutral. Mar 23, 2016 at 18:44
• That's right--and it's part of my point. You cannot provide a valid or useful answer without referring to these things, because they have a strong influence on what the answer should be. Whether or not you agree with this position, you certainly cannot justify an unqualified, universal pronouncement that region 1 is necessarily the right decision! (BTW, you're not getting any downvotes from me... .)
– whuber
Mar 23, 2016 at 18:48

my experience Use GraphPad (for example) and get the p-value for

                        8   2
500   500


You will p=0.1077 not significant at 5%. Try the same proportion for region 1 (80%) this time with 20 oservations

                       16   4
500   500


Surprised?

• _______20 obervations (of course...) Feb 20, 2016 at 5:29
• Is the Fisher´s Exact Test, you know? Feb 20, 2016 at 5:36
• Because the O.P. is asking for a decision, what decision are you actually recommending?
– whuber
Feb 22, 2016 at 14:27
• I was wondering, because the question asks about how to decide where to sell bananas. It's not clear whether or how your procedure is answering that question.
– whuber
Feb 24, 2016 at 19:36
• I think you are answering a different question. What my answer would be is irrelevant--but since you ask, if I were answering I would point out that a null hypothesis test involves a 0-1 loss function, but that a business decision like selling a product into a market will have a very different loss function. Thus, we should not expect the result of a hypothesis test to be relevant to the business decision.
– whuber
Feb 24, 2016 at 20:43