Accuracy of random prediction with non equal distribution Assume that I want to predict the value of a variable that has three different states: a, b, and c.
The chance that these variables have the 3 states is not equally distributed. Out of 10 trials, the distribution is like this:
a: 2
b: 3
c: 5
If I have a random predictor that randomly predicts whether the variable is a, b, or c, the chance are intuitively not 1/3. But what are the chances and how can I calculate them?
In addition, I have a follow up question. Assume these new distribution:
a: 1
b: 0
c: 9
Here one can see, that if my classifier just always predicts c, he is in 90% of the times correct. Is there any way to combine these two concepts? I want to calculate the accuracy of a classifier that predicts randomly, but only if it's not better to just predict always the same.
 A: This question is related to the old joke about how the stopped clock is more accurate than the slow clock because the stopped clock is correct twice each day.  There's truth to that, as we can demonstrate using the basic axioms of probability.

Let's solve the general problem: you observe a random variable $X$ that can attain the values $a_1, a_2, a_3, \ldots, a_n, \ldots,$ with specified probabilities $p_1, p_2, p_3,\ldots, p_n, \ldots,$ respectively, and you choose to predict it with an independent random variable $Y$ with associated probabilities $q_1, q_2, q_3, \ldots, q_n, \ldots,$ etc.  If you happened to know all the $p_n$, then what values should you choose for the $q_n$?
Stated this way, it becomes a simple optimization problem: let us maximize the chance of the event that $Y$ is correct, $X=Y$.  By enumerating the possible outcomes $a_1, a_2, \ldots, a_n, \ldots, $ which are mutually exclusive, the axioms of probability tell us this chance is obtained by summing over the outcomes:
$$\Pr(X=Y) = \sum_{n} \Pr(X=a_n,\ Y=a_n).$$
Because $X$ and $Y$ are independent, their probabilities multiply:
$$\Pr(X=a_n,\ Y=a_n) = \Pr(X=a_n)\Pr(Y=a_n)$$
for all $n$.  Putting these together yields
$$\Pr(X=Y) = \sum_{n} p_n q_n.$$
This answers the first question about how to compute the chances that $Y$ will predict $X$ correctly.  For instance, if you estimate that $X$ has $2/10$ chance to equal $a_1$, $3/10$ chance to equal $a_2$, and $5/10$ chance to equal $a_3$ and you let $Y$ be uniformly distributed across these three outcomes, then
$$\Pr(X=Y) = \frac{2}{10}\frac{1}{3} + \frac{3}{10}\frac{1}{3} + \frac{5}{10}\frac{1}{3} = \frac{1}{3}$$
is the chance that $Y$ will correctly predict $X$.
Given that the $p_n$ must sum to unity and so must the $q_n$, the solution can be found almost instantly using a host of techniques such as Lagrange multipliers or classical inequalities.  For instance, a simple (and obvious) version of the Hölder Inequality asserts 
$$\sum_{n} p_n q_n \le \max(p_n)\sum_n q_n = \max(p_n).$$
Clearly this upper bound is attained when $q_i = 1$ for a single $i$ for which $p_i = \max(p_n)$.  In other words, $Y$ should not be random at all: you should always predict that $X$ will equal one of its most frequent values.  This answers (what I believe to be) the second question.
Continuing the previous example, the most frequent value of $X$ is $a_3$, so make $Y$ always predict that $X=3$.  The chance it will be correct is $\Pr(X=3) = 5/10$, much larger than attained by the uniform random predictor.

To apply this solution to the clock joke, suppose that times are read to the nearest minute and let $X$ be a random time.  How best to predict $X$?  The answer is to predict any one of the most frequent times.  Since each of the $720$ distinct minutes shown on a clock has an equal chance (of $1/720$) of being correct, let $Y$ constantly equal any fixed one of these minutes.  It describes a clock that never moves; it has stopped with its hands pointing to that time.  Although there exist other predictors (e.g., randomly select among any number of stopped clocks), we have proven that they cannot be any better than one of these stopped clocks.
A: You answered your own question. If 90% of people fall into group A then the best "naive predictor" would be to assign everyone to group A. If you did this, you would be right 90% of the time. Any classification algorithm ought to be compared to this naive prediction rule.
