Some random reflections:

First, it works as a measure of "surprise" or uncertainty since $$\underset{p\to1}\lim p\log(p) = \underset{p\to 0}\lim p\log(p) = 0$$ and the curve is concave with a maximum below $p < 0.5$ as the probability decreases. In the case of a Bernoulli experiment:

The maximum entropy is under a uniform distribution $p=1/n$:

$$H(X)=\sum_{i=1}^{n}\frac 1 n\log(1/p)=\frac 1 n n\log(n)=\log(n)$$

and corresponds to the log of the number of events $n.$

Second, the Shannon information function $I(p) = - \sum_i \log(p_i)$ solves the Cauchy multiplicative functional equation $f(ab) = f(a) + f(b)$: the information of independent events $I(p_1·p_2) = I(p_1) + I(p_2)$ is the sum of the information of each event (see discussion here). This translates into the additivity property of Shannon's entropy: for two independent random variables $X$ and $Y,$ $H(X,Y) = H(X) + H(Y)$:

$$\begin{align}H(X,Y) &=-\sum_{x\in X}\sum_{y\in Y} p(x,y) \log_2\left(p(x)p(y)\right) \\\\ &= -\sum_{x\in X}\sum_{y\in Y} p(x)p(y) \log_2(p(x)) - \sum_{x\in X}\sum_{y\in Y} p(x)p(y)\log_2(p(y)) \\\\ &=H(X) + H(Y) \end{align}$$

Hence, the logarithm turns the multiplicative rule of probabilities in independent random variables into addition of their entropies.

Third, the presence of the logarithm results in a mirror definition of Boltzman entropy $S= k \log W$, in which $W$ represents the number of microstates configurations, much like the probabilities individual events.

Some random reflections:

First, it works as a measure of "surprise" or uncertainty since $$\underset{p\to1}\lim p\log(p) = \underset{p\to 0}\lim p\log(p) = 0$$ and the curve is concave with a maximum below $p < 0.5$ as the probability decreases. In the case of a Bernoulli experiment:

The maximum entropy is under a uniform distribution $p=1/n$:

$$H(X)=\sum_{i=1}^{n}\frac 1 n\log(1/p)=\frac 1 n n\log(n)=\log(n)$$

and corresponds to the log of the number of events $n.$

Second, the Shannon information function $I(p) = - \sum_i \log(p_i)$ solves the Cauchy multiplicative functional equation $f(ab) = f(a) + f(b)$: the information of independent events $I(p_1·p_2) = I(p_1) + I(p_2)$ is the sum of the information of each event (see discussion here). This translates into the additivity property of Shannon's entropy: for two independent random variables $X$ and $Y,$ $H(X,Y) = H(X) + H(Y)$:

$$\begin{align}H(X,Y) &=-\sum_{x\in X}\sum_{y\in Y} p(x,y) \log_2\left(p(x)p(y)\right) \\\\ &= -\sum_{x\in X}\sum_{y\in Y} p(x)p(y) \log_2(p(x)) - \sum_{x\in X}\sum_{y\in Y} p(x)p(y)\log_2(p(y)) \\\\ &=H(X) + H(Y) \end{align}$$

Hence, the logarithm turns the multiplicative rule of probabilities in independent random variables into addition of their entropies.

Third, the presence of the logarithm results in a mirror definition of Boltzman entropy $S= k \log W$, in which $W$ represents the number of microstates configurations, much like the probabilities individual events.

Some random reflections:

First, it works as a measure of "surprise" or uncertainty since $$\underset{p\to1}\lim p\log(p) = \underset{p\to 0}\lim p\log(p) = 0$$ and the curve is concave with a maximum below $p < 0.5$ as the probability decreases. In the case of a Bernoulli experiment:

The maximum entropy is under a uniform distribution $p=1/n$:

$$H(X)=\sum_{i=1}^{n}\frac 1 n\log(1/p)=\frac 1 n n\log(n)=\log(n)$$

and corresponds to the log of the number of events $n.$

Second, the Shannon information function $I(p) = - \sum_i \log(p_i)$ solves the Cauchy multiplicative functional equation $f(ab) = f(a) + f(b)$: the information of independent events $I(p_1·p_2) = I(p_1) + I(p_2)$ is the sum of the information of each event (see discussion here). This translates into the additivity property of Shannon's entropy: for two independent random variables $X$ and $Y,$ $H(X,Y) = H(X) + H(Y)$:

$$\begin{align}H(X,Y) &=-\sum_{x\in X}\sum_{y\in Y} p(x,y) \log_2\left(p(x)p(y)\right) \\\\ &= -\sum_{x\in X}\sum_{y\in Y} p(x)p(y) \log_2(p(x)) - \sum_{x\in X}\sum_{y\in Y} p(x)p(y)\log_2(p(y)) \\\\ &=H(X) + H(Y) \end{align}$$

Hence, the logarithm turns the multiplicative rule of probabilities in independent random variables into addition of their entropies.

Third, the presence of the logarithm results in a mirror definition of Boltzman entropy $S= k \log W$, in which $W$ represents the number of microstates configurations, much like the probabilities individual events.

Some random reflections:

First, it works as a measure of "surprise" or uncertainty since $$\underset{p\to1}\lim p\log(p) = \underset{p\to 0}\lim p\log(p) = 0$$ and the curve is concave with a maximum below $p < 0.5$ as the probability decreases. In the case of a Bernoulli experiment:

The maximum entropy is under a uniform distribution $p=1/n$:

$$H(X)=\sum_{i=1}^{n}\frac 1 n\log(1/p)=\frac 1 n n\log(n)=\log(n)$$

and corresponds to the log of the number of events $n.$

Second, the Shannon information function $I(p) = - \sum_i \log(p_i)$ solves the Cauchy multiplicative functional equation $f(ab) = f(a) + f(b)$: the information of independent events $I(p_1·p_2) = I(p_1) + I(p_2)$ is the sum of the information of each event (see discussion here). This translates into the additivity property of Shannon's entropy: for two independent random variables $X$ and $Y,$ $H(X,Y) = H(X) + H(Y)$:

$$\begin{align}H(X,Y) &=-\sum_{x\in X}\sum_{y\in Y} p(x,y) \log_2\left(p(x)p(y)\right) \\\\ &= -\sum_{x\in X}\sum_{y\in Y} p(x)p(y) \log_2(p(x)) - \sum_{x\in X}\sum_{y\in Y} p(x)p(y)\log_2(p(y)) \\\\ &=H(X) + H(Y) \end{align}$$

Hence, the logarithm turns the multiplicative rule of probabilities in independent random variables into addition of their entropies.

Third, the presence of the logarithm results in a mirror definition of Boltzman entropy $S= k \log W$, in which $W$ represents the number of microstates configurations, much like the probabilities individual events.