How should I measure my confidence in this estimate of this finite distribution Suppose I have a six sided die, that may not be fair. The probabilities for the six sides are equally likely to be any values in the interval [0,1], subject to the constraint that they must add up to 1. Suppose I wish to estimate this finite distribution. One plausible option is that after n throws of the die, I should estimate p1 as number of 1s rolled over n, and so on up to p6.
If I were to proceed in this way, how should I measure/quantify my confidence in this estimate of this finite distribution?
If I don't proceed this way, what other approaches are there to estimate the distribution on the basis of the observed evidence? Should I start from the assumption that the die is fair?
 A: EBartrum I have a few comments and hopefully some answers.
First of all when you say "the probabilities for the six sides are equally likely to be any values in the interval [0,1], subject to the constraint that they must add up to 1," unfortunately this does not make sense.  If you are looking for a way of describing a distribution on dice (Multinomial Distributions) you may want to look into the Dirichlet Distribution.  Also, if you're interested in thinking about the probability of different dice, as opposed to viewing your die as a fixed entity with fixed parameters, you may want to consider Bayesian Credible Intervals instead.


*

*All of that being said, if you are focusing on one of the sides (for example you focused your comments on $p_1$) then your die might as well be a coin, that either lands on side 1 or does not.  In that sense you can obtain confidence intervals based on the usual confidence intervals of proportions.  That being said do not construct multiple confidence intervals for the different faces and expect them to behave independently, because one interval not containing the true value of the parameter will increase the probability that another does not.

*If you would like to consider and confidence "region" as opposed to an interval, there is apparently some study of this that can be found by searching "Multinomial Confidence Interval".  This reference gives you some sense of the asymptotic error of the estimates you suggested http://www.math.wsu.edu/faculty/genz/papers/mvnsing/node8.html.

*Lastly, I suggest you look into the Dirichlet-Multinomial Distribution.  The Bayesian approach to probability is focused on modelling our uncertainty about certain parameters, and can be much more general and interpretable than confidence intervals.
Edit
Yes there is precisely such an approach.  Let me give you the rough sketch and hopefully enough jargon so that you can continue.
A Dirichlet Distribution of order $k \geq 2$ is a distribution that is only positive on the $(k-1)$-simplex.  This simplex is precisely the set of $k$-dimensional vectors with positive coordinates that sum to 1.  When you say you want to start off with some uniform assumption, what you want is a Uniform Distribution on this simplex, which happens to be a special case of the Dirichlet Distribution.  The Dirichlet Distribution is closely related to the Multinomial Distribution because it is conjugate prior.  This means that if you assume your die came from a certain Dirichlet Distribution, and then you observe data (roll your die) and updat your belief (distribution) about where your die came from, you still get a Dirichlet.
So to give you an idea:


*

*Start with a uniform distribution $Dir(6, (1,1,1,1,1,1))$ on 6-sided die.

*After seeing $n_i$ tosses for each face you update to $Dir(6, (n_1+1, \dots, n_6+1))$.  You still have a distribution modelling your uncertainty.

*You can ask and answer questions about this distribution, what is the mean, mode, variance (the mean might surprise you).

*This distribution will "converge" to the true die in the sense that the mean and mode will converge to the true die, and that the probability mass of any open neighbourhood of the true die will tend to 1.


I hope this was enough to get you going.  A special case of this is the Beta-Binomial model.  If you have any more questions let me know.
