What is the significance of the log base being 2 in entropy? What is the significance of the log base being 2 in entropy? What if we take e or 10 as the base?
 A: I'd like to cite that in the Elements of Information Theory by Covers:

If the base of the logarithm is b, we denote the entropy as $H_b(X)$.If
  the base of the logarithm is e, the entropy is measured in nats.Unless
  otherwise specified, we will take all logarithms to base 2, and hence
  all the entropies will be measured in bits.

And in lemma 2.1.2: 

$H_b(X) = (log_b a)H_a(X)$ Proof:
  $log_bp = log_ba\ log_ap$.
The second property of entropy enables us to change the base of the
  logarithm in
  the definition. Entropy can be changed from one base to another by
  multiplying by the appropriate factor.

Hope it helps. 
A: Nothing much. Recall that we have $$\log_a b = \frac{\log_c b}{\log_c a}$$
Hence, if you use other base, such as $e$ and $10$, you can always convert to another base using a scalar multiplication. 
Communication/ information was thought in terms of bits, hence the magical number, $2$.
A: Apart from the fact that using bits (letters or symbols that can have one of the pre-determined 2 values at a time) is the most convenient way of encoding data, there is no other significance. If there were no computers or IT, we might as well be using $\log_{10}$, and in that case, we would have needed much less number of letters (digits) for encoding. Or, a combination of all digits and Roman alphabets would have brought $\log_{36}$ into picture.
And as stated in another answer, the conversion is straightforward.
But what is slightly more perplexing is the idea that our choice for the number of values the letters can have is not limited to natural numbers! In fact, using an $e$-valued letter is the most natural choice and such a letter is called a nat or natural unit of information. Again the conversion is straightforward. But if you wanted to go deeper, have a look at Information Theory - Rationale Behind Using Logarithm for Entropy, and Other Explanations.
