2
$\begingroup$

How can you classify the accuracy of a probability?

Say I do a study of people that like bananas in 2 different regions.

  • Region 1: 8 out of 10 people tested like bananas.
  • Region 2: 500 out of 1000 people tested like bananas.

So in region 1, 80% of people like bananas. In region 2, 50% of people like bananas. I have a dilemma if want to sell bananas in either region 1 or 2. The accuracy of the probability in region 1 will be worse than in region 2. How can I quantify this accuracy to make a decision, in which region should I sell bananas?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ I think you're confusing the concepts of accuracy and precision. Without any additional information, we don't know whether 50% or 80% is the more accurate estimate. We can say, however, based on the sample size that the precision of the 50% estimate is much higher and thus our variability is much lower. $\endgroup$
    – Matt Brems
    Feb 20, 2016 at 0:08

3 Answers 3

Reset to default
1
$\begingroup$

This sounds like this question could be rephrased as "Is the proportion of people who like bananas in Region 1 significantly different than in Region 2?"

Reframed that way, the statistical test would be for the z-test for the difference between two proportions.

Here is a link to an online calculator: http://epitools.ausvet.com.au/content.php?page=z-test-2

Note, however, that Region 1 (at least in your example), is undersampled, and the statistical tests may be inaccurate. Many sources recommend sampling until you have at least 10 people who meet each condition (in this example, 10 people who like and 10 who dislike bananas in each region).

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I like your reasoning. But why not recommend using a test that is suitable for the data (such as a Binomial test) rather than suggesting that more samples should be collected? $\endgroup$
    – whuber ♦
    Feb 22, 2016 at 14:25
-1
$\begingroup$

In this particular case, accuracy/precision and other sensible considerations don't matter. If the information that you gave in the question is all you know, then you have to sell bananas in region 1. Period.

However, if the price of bananas in region 2 was higher than in region 1, it would have been a different story. The reason is that the price asymmetry would bring the cost of error asymmetry.

What if in reality the region 2 liked bananas more than region 1? In this case although given the data it seems that region 1 has higher propensity to buy bananas, depending on the price differential the cost of error could be too high given the variances of your estimates.

UPDATE:

@whuber brought up an interesting topic: risk aversion. If you're averse to risks, then you may consider expected utility theory, it's studied in microeconomics and game theory. The trouble is that there's not enough data to set up the expected utility function.

For instance, in finance the simple examples are usually like follows. You have two stocks. Stock 1 returns 80% annually with standard deviation 60%, Stock 2 returns 50% annually with standard deviation 5%. Which stock to choose?

You need a utility function, such as $U(\mu,\sigma)=\mu-\sigma^2$. In this case $U_1=44%$ and $U_2=0.4975$., so you have to go with the Stock 2 despite its average return us lower, because it's risk is "too high".

Improve this answer
$\endgroup$
5
  • $\begingroup$ I think anybody who is aware of and concerned about utility and risk aversion in decision making would strongly disagree with your opening paragraph. Region 1 is a riskier prospect because the information about it is less certain. That issue should not dismissed out of hand. Don't you suppose this is the very reason the question was posed? $\endgroup$
    – whuber ♦
    Mar 23, 2016 at 18:11
  • $\begingroup$ @whuber, why is Region 1 riskier? Is it because of your prior knowledge that generally people dislike bananas? Based on the information in the question there's nothing to suggest that Region 1 should have lower than 50% people liking bananas. $\endgroup$
    – Aksakal
    Mar 23, 2016 at 18:25
  • $\begingroup$ Consider a Bayesian analysis (because it's easier to explain) and suppose you adopt a weak conjugate prior of a $B(1,1)$ distribution on the proportion $p$ of people who will buy bananas. To oversimplify, suppose further that (1) you cannot tolerate a loss and (2) losses are likely when $p\le 0.45$. Then in region (1) the posterior chance of a loss is $3.6\%$ while in region (2) the posterior chance is only $0.08\%$. This reflects risk. A more nuanced analysis would use a more realistic loss function, incorporate utility, and assess the sensitivity to the assumed prior distribution. $\endgroup$
    – whuber ♦
    Mar 23, 2016 at 18:33
  • $\begingroup$ @whuber, but you brought in a lot of stuff which was not in OP's question. He didn't indicate any loss aversion. He might be risk seeking or neutral. $\endgroup$
    – Aksakal
    Mar 23, 2016 at 18:44
  • $\begingroup$ That's right--and it's part of my point. You cannot provide a valid or useful answer without referring to these things, because they have a strong influence on what the answer should be. Whether or not you agree with this position, you certainly cannot justify an unqualified, universal pronouncement that region 1 is necessarily the right decision! (BTW, you're not getting any downvotes from me... .) $\endgroup$
    – whuber ♦
    Mar 23, 2016 at 18:48
-5
$\begingroup$

my experience Use GraphPad (for example) and get the p-value for 

                        8   2
                      500   500

You will p=0.1077 not significant at 5%. Try the same proportion for region 1 (80%) this time with 20 oservations

                       16   4
                      500   500

Surprised?

Improve this answer
$\endgroup$
10
  • $\begingroup$ _______20 obervations (of course...) $\endgroup$
    – licas
    Feb 20, 2016 at 5:29
  • $\begingroup$ Is the Fisher´s Exact Test, you know? $\endgroup$
    – licas
    Feb 20, 2016 at 5:36
  • 1
    $\begingroup$ Because the O.P. is asking for a decision, what decision are you actually recommending? $\endgroup$
    – whuber ♦
    Feb 22, 2016 at 14:27
  • 2
    $\begingroup$ I was wondering, because the question asks about how to decide where to sell bananas. It's not clear whether or how your procedure is answering that question. $\endgroup$
    – whuber ♦
    Feb 24, 2016 at 19:36
  • 4
    $\begingroup$ I think you are answering a different question. What my answer would be is irrelevant--but since you ask, if I were answering I would point out that a null hypothesis test involves a 0-1 loss function, but that a business decision like selling a product into a market will have a very different loss function. Thus, we should not expect the result of a hypothesis test to be relevant to the business decision. $\endgroup$
    – whuber ♦
    Feb 24, 2016 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.