0
$\begingroup$

If I have a hierarchy of information where the root node represents all possible outcomes and each child refers to a more specific outcome is the child node a higher or lower entropy than the root node?

Hierarchy example: root node is (person) and child nodes are (male person) and (female person).

I'm confused because the root (person) could be male or female so it represents all the information combined from each child.. meaning it holds the most information.. but each child has more specific information than the root.. so they each have more information about the person.. which has more information? Which would I call a higher entropy node in the sense of information entropy, the root or the child?

Thanks!

$\endgroup$
0
$\begingroup$

Entropy is not quantification of amount of information. Entropy is quantification of surprise (or unpredictability) in the information.

In your example, the top of node actually has less predictable information. It says that you will either see a male or a female, which is only somewhat helpful. On the other hand, once you move to a particular child node (say right one), you don't expect a surprise (you expect to see, say, a female). Since the surprise (or unpredictability) has gone down, the entropy of the system is lower.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.