Suppose you have a discrete random variable $X_1$ with known probability mass function. I guess that choosing a variable drawn from the same pmf would be the best way to guess $X_1$ assuming all experiments are independent. If this is true, what is the name of this theorem?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
By the most usual criteria of "best guess", that's not an optimal strategy. For example:
If you want to miminize average square error : pick E(X)
If you want to miminize average absolute error : pick median(X)
If you want to miminize probability of error: pick mode(X) (maximum value of pmf)
To use random strategies for guessing can be optimal is some scenarios where there is an opponent who can vary his strategy observing yours (eg. rock-paper-scissors), but that doesn't seem your scenario. Actually I got this question in mathstackexchange but I thought it is suitable for users on this site.
$\begingroup$
$\endgroup$
The interpretation of best way to guess given in the previous answer assumes you are interested in estimating a single summary statistic (eg. mean).
If instead you are interested in the entire distribution you can use this approach of sampling to build up the Empirical Cumulative Distribution Function: if your samples are $x_1,\ldots, x_n$ this is the function given by
$$ F(u) = \frac{1}{n} \sum_i \mathbf 1_{x_i \leq u},$$
i.e. for any $u$ it is the proportion of observed samples that are smaller than $u$.
The Glivenko-Cantelli theorem demonstrates that this converges to the CDF of the original variable.
The exact form of convergence (almost sure convergence in $L_\infty$) will require some understanding of advanced probability.
