I'd like to add a straightforward answer to this question:

what does that quantity actually tell me?

It's intuitive to illustrate that in a discrete scenario. Suppose that you toss a heavily biased coin, saying the probability of seeing a head on each flip is 0.99. Every actual flip tells you very little information because you almost already know that it will be head. But when it comes to a fairer coin, it't harder for you to have any idear what to expect, then every flip tells you more information than any more biased coin. The quantity of information obtained by observing a single toss is equated with $\log \frac{1}{p(x)}$.

What the quantity of the entropy tells us is the information every actual flipping on (weighted) average can convey: $E \log \frac{1}{p(x)} = \sum p(x) \log \frac{1}{p(x)} $. The fairer the coin the larger the entropy, and a completely fair coin will be maximally informative.

what does that quantity actually tell me?

I'd like to plug in a straightforward answer as follows:

It's intuitive to illustrate that in a discrete scenario. Suppose that you toss a heavily biased coin, saying the probability of seeing head on each flip is 0.99. Every actual flip tells you very little information because you almost already know that it will be head. But when it comes to a fairer coin, it's harder for you to have any idea what to expect, then every flip tells you more information than any more biased coin. The quantity of information obtained by observing a single toss is equated with $\log \frac{1}{p(x)}$.

What the quantity of the entropy tells us is the information every actual flipping on average(weighted by its probability of occuring) can convey: $E \log \frac{1}{p(x)} = \sum p(x) \log \frac{1}{p(x)} $. The fairer the coin the larger the entropy, and a completely fair coin will be maximally informative.

I'd like to add a straightforward answer to this question:

what does that quantity actually tell me?

It's intuitive to illustrate that in a discrete scenario. Suppose that you toss a heavily biased coin, saying the probability of seeing a head on each flip is 0.99. Every actual flip tells you very little information because you almost already know that it will be head. But when it comes to a fairer coin, it't harder for you to have any idear what to expect, then every flip tells you more information than any more biased coin.

What the quantity of the entropy tells us is the information every actual flipping on average can convey. The fairer the coin the larger the entropy, and a completely fair coin will be maximally informative.

I'd like to add a straightforward answer to this question:

what does that quantity actually tell me?

It's intuitive to illustrate that in a discrete scenario. Suppose that you toss a heavily biased coin, saying the probability of seeing a head on each flip is 0.99. Every actual flip tells you very little information because you almost already know that it will be head. But when it comes to a fairer coin, it't harder for you to have any idear what to expect, then every flip tells you more information than any more biased coin. The quantity of information obtained by observing a single toss is equated with $\log \frac{1}{p(x)}$.

What the quantity of the entropy tells us is the information every actual flipping on (weighted) average can convey: $E \log \frac{1}{p(x)} = \sum p(x) \log \frac{1}{p(x)} $. The fairer the coin the larger the entropy, and a completely fair coin will be maximally informative.