2
$\begingroup$

I cannot figure out this simple entropy problem and it is driving me crazy!

From McElreath's Statistical Rethinking:

Imagine instead 5 buckets and a pile of 10 individually numbered pebbles. You stand and toss all 10 pebbles such that each pebble is equally likely to land in any of the 5 buckets.

Distribution & Entropy:

A <- (0,0,10,0,0) = 0

B <- (0,1,8,1,0) = 0.6390319

C <- (0,2,6,2,0) = 0.9502705

D <- (1,2,4,2,1) = 1.4708085

E <- (2,2,2,2,2) = 1.6094379

For "B" I did -1[(.1)log2(.1) + (.8)log2(.8) + (.1)log2(.1)] = 0.92

For "E" I did -1[(.2)log2(.2) + (.2)log2(.2) + (.2)log2(.2) + (.2)log2(.2) + (.2)log2(.2)] = 2.32

Where did I go wrong?!?!

$\endgroup$

1 Answer 1

1
$\begingroup$

It's because the entropy in this exercise is calculated using natural logarithm. For example, for (B), we will have:

$$\mathcal H_B=-2\times 0.1\times \ln(0.1)-0.8\times \ln(0.8)\approx 0.6390$$

$\endgroup$
1
  • 1
    $\begingroup$ Thank you so much for your response. I greatly greatly appreciate it. $\endgroup$
    – CuriousMD
    Commented Dec 29, 2020 at 21:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.