Return to Revisions

Here's an off-the-cuff explanation. You could say 2 books of the same size have twice as much information as 1 book, right? (Considering a book to be a stream of bits.) Well, if a certain outcome has probability P, then you could say its information content is about the number of bits you need to to write out 1/P. (e.g. if P=1/256, that's 8 bits.) Entropy is just the average of that information bit length, over all the outcomes.