# Which is a better baseline - Probabilistic Assignment vs Deterministic Assignment

Problem Setup: Let's say a bag contains 100 balls, 60 of which are red and 40 are black. 100 Balls are drawn at random from the bag (with replacement) and before each draw, you have to guess whether the drawn ball will be red or black.

Solution 1: You always guess that the drawn ball will be Red. This will result in an expected accuracy of 0.6

Solution 2: You flip a biased coin with $P(Heads) = 0.6$ and if the coin lands head, you guess that the drawn ball will be Red otherwise Black. Please note that the probability of 0.6 was chosen because the $P(red ball)$ in the bag is 0.6

Question for the Cross Validated Community: What is the expected accuracy for solution 2?

PS: This question was marked to be Closed because it was not clear what was being asked. Therefore, I have completely reworded the problem and the scenario and have explained it in as simpler way as possible.

• You'll have to say a bit more about what you are trying to accomplish with this "assignment." If you already know which cases have cancer, it's not clear that there's any need for "assignment."
– EdM
Dec 25, 2017 at 19:58
• @EdM: Generally there is a concept of training set and test set. Also one assumes that both training and test set follow the same distribution. P(cancer) = 0.6 was determined by looking at the training set and then for every case you see in the test set, you have to determine whether it is a case of cancer or not. So my "assignment" is for the cases in the test set. Dec 25, 2017 at 20:27
• @EdM: Let me also briefly introduce the concept of baselines. Baselines are meant to be some really simple way of accomplishing the task. One generally compares "fancy methods" with the performance of baseline methods. Dec 25, 2017 at 20:30
• Is there a reason you need to hard assign each case to cancer or not cancer? Often a better idea is to use a proper scoring rule loss function like log-loss, which measures how well the probability agrees with the class. It is really only in rare cases where accuracy is a reasonable evaluation metric. Dec 25, 2017 at 22:03
• @MatthewDrury: Evaluation metric is not the main concern here. I am fine using Precision and Recall as well. My concern is, how do I measure that for the case of the probabilistic assignment? Dec 25, 2017 at 22:46

You're combining two independent events, so their joint probability distributions can be split: \begin{align} P(Correct) &= P(Red_{Ball},Red_{Coin}) + P(Black_{Ball},Black_{Coin}) \\ &= P(Red_{Ball})\cdot P(Red_{Coin}) + P(Black_{Ball})\cdot P(Black_{Coin}) \\ &= 0.6 \cdot 0.6 + 0.4\cdot 0.4 \\ &= 0.52 \end{align}