Ok, no maths. I too was curious to know the same thing, and have understood the purpose served by using logarithm in entropy equation?.

Let's use a simple example. Suppose we are looking at 3 different containers. Each container has some triangles or circles.

Let's focus on first container - Container 1 which has 26 triangles, 4 circles. If you put your hand inside the container and picked one, then what is the chance that you pick a triangle (or a circle)?

If you were allowed to pick until there is nothing left in Container 1, that would mean after 30 picks you would have 26 triangles, 4 circles in hand. So you can say "The chance of getting a triangle is 26/30 and not getting a triangle is 4/30". The exact opposite can be said about picking a circle.

In other words, because the number of triangles is more than the number of circles there is a higher chance (likelihood/probability/confidence) that you will pick a triangle when you take something out of Container 1. This likelihood is what we usually term a probability.

To put it another way, when picking one item at a time randomly from Container 1, you have less doubt (uncertainty/surprise) about picking a triangle. In the same way, you have more doubt (less certain) about picking a circle. This uncertainty is what is usually termed as entropy, which is the opposite of probability.

That means, for Container 1 you had higher ~87% chance to pick a triangle (26/30), less chance ~13% to pick a circle. You have more confidence you'll pick a triangle but if you do get a circle at times, you won't be as surprised because chances of getting a circle are less.

In other words, entropy (element of surprise, chance that you get something else than what you expected) will be less for Container 1.

Similarly, Container 2 has a ~50% chance to pick a triangle (14/30), ~50% chance for circle (16/30).

To use the same line of thought, from Container 2 it is equally likely you pick a triangle or circle on any pick, you have same confidence about picking(or same uncertainty about not picking) a triangle or a circle. So entropy (element of surprise, chance that you will get something else than what you expected) is higher for Container 2.

Finally for Container 3 no chances you'll pick a triangle (0/20), guaranteed chance you'll pick a circle (20/20). Thus, whenever you pick something out, you have absolutely no doubt (entropy is zero for Container 3) that it will be a circle.

What purpose does the logarithm serve in entropy equation?

How can one put a number on probability, entropy for Container 1 (or Container 2, Container 3) as a whole? It would be ideal to have the number between 0 and 1 so that one could easily represent it as a percentage.

Looking at Container 1, the ratio of number of triangles to the total number of items is 26/30 (~0.87). For circles it is 4/30 (~0.13). Trying to just add these two ratios doesn't make sense (because I get back 30/30 = 1). I need some way to scale them (make them bigger or smaller) between 0 and 1. This scale should be the same irrespective of the number of items e.g. the same scale should work for Container 3 even if it has different number of items compared to Containers 1, 2.

Logarithm provides this scale. Logarithms offer a way to represent numbers (especially if they are large) into reduced (or scaled down) versions. If multiply 26/30 with log(26/30) then I can scale it. To ensure my scale stays independent (because total numbers of items can vary in different containers), one option is to use 2 as the base for logarithms. Any base (2, 10, e) will suffice as long as one measures each container using the same base. Specific to our examples, we either a triangle or we don't - so usually we have 2 choices. Base 2 represents 0, 1 as 2 choices in computers hence that has been the choice of base to use for logarithm.

Using this fact about logarithms, if we multiply each ratio with it's log, we effectively scale it between 0 and 1.

So using that background we can calculate entropy for each container as follows

Note: Since we divide by log(total number of items) to keep the new scaled ratio between 0 and 1, multiplying by that ends up being less than 0 (hence the negative sign is added to make it back to positive).

As you can see, entropy (doubt, surprise, uncertainty) for Container 1 is less (56%), but more for Container 2 (99%). For Container 3, there is no (0%) entropy - you have 100% chance to pick a circle.

Hope this explanation helps to both visualize and make an intuition about Entropy, the choice of logarithms - statistical functions almost always use logarithms to scale down ratios and large numbers towards a common range representable between 0 and 1.