Over a sample (of about a hundred observations), I have estimated the "empirical" entropy of a sequence of identically independently distributed random variables which have 5 possible outcomes.

I was curious to know if there was a possibility to estimate in some way a confidence interval over this "empirical" entropy calculation (i.e. where true probabilities are replaced by their observed frequencies) ?

Best regards

  • $\begingroup$ I've made a few edits to clear up your question a little, but I couldn't work out what to do with the singular/plural mismatches in "a repeated i.i.d. random variables"; please edit to clarify. $\endgroup$
    – Glen_b
    Apr 8 '14 at 9:33
  • $\begingroup$ You can compute a confidence interval of most things by bootstrapping. Does that help? $\endgroup$ Apr 8 '14 at 9:37
  • $\begingroup$ @ Glen_b: thank's for the edit I have modified the question hopefully in accordance with your comment. Best regards $\endgroup$
    – TheBridge
    Apr 8 '14 at 12:55
  • $\begingroup$ @ Adam Ryczkowski : I was more looking at some "asymptotic normality" kind of results, but this approach might work, I have to gie ti a try. Best regards $\endgroup$
    – TheBridge
    Apr 8 '14 at 12:56
  • $\begingroup$ (When referring to someone with the @ sign make sure that there is no space between it and the user's name. Otherwise SE ignores it and people do not get alerted about the ongoing conversation). $\endgroup$ Apr 8 '14 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.