To maximize the chance of correctly guessing the result of a coin flip, should I always choose the most probable outcome? This is not homework. I am interested in understanding if my logic is correct with this simple stats problem.
Let's say I have a 2 sided coin where the probability of flipping a head is $P(H)$ and the probability of flipping a tail is $1-P(H)$. Let's assume all flips have independent probabilities. Now, let's say I want to maximize my chances of predicting whether the coin will be a head or tail on the next flip. If $P(H) = 0.5$, I can guess heads or tails at random and the probability of me being correct is $0.5$.
Now, suppose that $P(H) = 0.2$, if I want to maximize my chances of guessing correctly, should I always guess tails where the probability is $0.8$?
Taking this one step further, if I had a 3-sided die, and the probability of rolling a 1, 2, or, 3 was $P(1)=0.1$, $P(2)=0.5$, and $P(3)=0.4$, should I always guess 2 to maximize my chances of guessing correctly? Is there another approach that would allow me to guess more accurately?
 A: You're right. If $P(H) = 0.2$, and you're using zero-one loss (that is, you need to guess an actual outcome as opposed to a probability or something, and furthermore, getting heads when you guessed tails is equally as bad as getting tails when you guessed heads), you should guess tails every time.
People often mistakenly think that the answer is to guess tails on a randomly selected 80% of trials and heads on the remainder. This strategy is called "probability matching" and has been studied extensively in behavioral decision-making. See, for example,
West, R. F., & Stanovich, K. E. (2003). Is probability matching smart? Associations between probabilistic choices and cognitive ability. Memory & Cognition, 31, 243–251. doi:10.3758/BF03194383
A: Due to independence your expectation value is always maximized if you guess the most likely case. There isn't a better strategy because each flip/roll doesn't give you any additional information about the coin/die. 
Anywhere you guess a less likely outcome your expectation of winning is less than if you had guessed the most likely case, thus you are better off just guessing the most likely case.
If you wanted to make it so that you did need to change your strategy as you flipped you might consider a coin/die where you don't know the odds initially and you have to figure them out as you roll.
A: You are essentially asking a very interesting question: should I predict using "MAP Bayesian" Maximum a posteriori estimation or "Real Bayesian".
Suppose you know the true distribution that $P(H)=0.2$, then using the MAP estimation, suppose you want to make 100 predictions on next 100 flip outcomes. You should always guess the flip is tail, NOT guessing $20$ head and $80$ tail. This is called "MAP Bayesian", basically you are doing 
$$\arg\max_ \theta f(x|\theta)$$
It is not hard to prove that by doing so you can minimize the predicted error (0-1 loss). The proof can be found in ~page 53 of Introduction to Statistical Learning.

There is another way of doing this called "Real Bayesian" approach. Basically you are not trying to "select the outcome with highest probability, but consider all the cases probablistically" So, if someone ask you to "predict next 100" flips, you should pause him/her, because when you given 100 binary outcomes, the probabilistic information for each outcome disappears. Instead, you should ask, what you want to do AFTER knowing the results. 
Suppose he/her has some Loss Function (not necessary to 0-1 loss, for example, the loss function can be, if you miss a head, you need to pay \$1, but if you miss a tail, you need to pay \$5, i.e., imbalanced loss) on your prediction, then you should using your knowledge about the outcome distribution to minimize loss over the whole distribution 
$$\sum_x \sum_y p(x,y) L(f(x),y)$$
, i.e., incorporate your knowledge about the distribution to loss, instead of "stage-wised way", getting the predictions and do next steps.
What's more, you have a very good intuition about what will have when there are many possible outcomes. MAP estimation will not work well if the number of outcome is large and the probability mass is widely spread. Think about you have a 100 side-dice, and you know the true distribution. Where $P(S_1)=0.1$, and $P(S_2)=P(S_3)=P(S_{100})=0.9/99=0.009090$. Now what you do with MAP? You will always guess you get the first side $S_1$, since it has largest probability comparing to others. However you will get wrong $90\%$ of the times!!
