# Information gain of flipping coins

I'm studying for an exam on Bishop's Pattern Recognition and Machine Learning (ISBN: 978- 0387310732). One of the questions in the mock exam paper is:

Three fair coins are flipped sequentially; you win if all three land with the same side up.

i) What is the expected information gain from seeing what side is up in the first coin?

ii) What is the information gain from also seeing the second coin?

I found Bishop's explanation confusing and it is not entirely clear to me what information gain actually represents regardless of its formula.

As far as I have understood the formula for calculated information gain is:

information=-log(probability)

and information gain is the degree of surprise of learning the value of x.

this is not entirely clear to me. My attempt to answer would be:

i) 0.301

ii) 0.301 + 0.301

Thank you

• Please add the self-study tag and read its tag-wiki, modifying your question to follow the guidelines there. – Glen_b -Reinstate Monica May 3 '15 at 12:15

The information of an event is defined as: ${I} (p)=\log(1/p)$ The probability of the first coin showing head is 0,5. Therefore the information gain is 0,301.