Suppose there is some distribution $F$ not known to us. However, we can get information about this distribution by means of samples, i.e. we have a set of random variables from this distribution. Ideally, I would like to quantify information I have about this distribution.

For example, if we have a coin with bias $p$, then after (sufficiently large) $N$ tosses we would have much lesser uncertainty about the value of $p$ than say if we had access to only one toss. I want to quantify this uncertainty.

During my general search for a related concept, I came across the idea of "entropy" in information theory. (btw, I did little bit of information theory in school and since then I have not used it once). This is very similar to what I want to do but entropy is defined for the source... or a single random variable. I could not find anything for a set of random variables.


A practical way of assessing properties of an estimator is the bootstrap; this seems to align with your goal. See http://en.wikipedia.org/wiki/Bootstrapping_(statistics) Bootstrapping uses resampling to assess the properties of estimators instead of more elaborate derivations; very practical.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.