Why is entropy sometimes written as a function with a random variable as its argument? Why do people sometimes write the entropy as a function on a random variable?
For example, in some class notes I am seeing:
$$ \sum_X Q(X) \log Q(X) = - H_Q(X) $$
I realize this is not to be taken as super formal since there is abuse of notation mixing the random variable and its value, but even after taking that into account this doesn't seem to make a lot of sense because the entropy is a function of the distribution $Q$ alone. Why not simply write $H_Q$ or even $H(Q)$? Why keep the $X$ around?
 A: It makes sense if you remember that X is a statistical ensemble, which has associated with it some probability distribution. For instance, the Wikipedia article for information-theoretic entropy consistently uses the notation $H(X)$. In that sense, it’s actually the Q that’s extraneous.
I can speculate two reasons your instructor includes the Q there.
The Q could be there to remind you what the associated probability distribution is. It could also be a preface to introducing concepts like cross-entropy, where you may take your average over a different distribution P. (The cross-entropy Wikipedia article exclusively uses distributions over the same support as arguments to the cross-entropy function.)
A: This is notation that bypasses the distribution (similar to moments)
As can be seen from the formula, the entropy is fully determined by the probability distribution of the random variable, not the random variable itself.  In the formula you give in your question the object $X$ is used as an index of summation, so the summation does not depend on $X$ (it is "summed out" of the equation).  (Note that it is bad notation to use the same notation for the random variable and the index of summation.)  The notation you are referring to is similar to the notation for the moments of random variables, which are also fully determined by the distribution of those random variables.  I will try to explain the alternative notational methods here.

Two possible notations: If you take $Q$ to denote the mass function of the random variable (and let $\mathscr{X}$ denote its support) then a natural notation for the entropy would be:
$$H(Q) \equiv - \sum_{x \in \mathscr{X}} Q(x) \log Q(x).$$
However, if you accept the notation for moments, you could also reasonably denote the entropy as:
$$H(X) \equiv \mathbb{E}(-\log(X)) = - \sum_{x \in \mathscr{X}} Q(x) \log Q(x).$$
These two alternatives frame the notation either as a function of the distribution, or a function of the random variable.  In the latter case we use the same notational convention used for moments of random variables, which are also functions of the underlying distribution of the random variable.

Technical issues with moment notation: From a technical perspective, a random variable is a mapping on the sample space.  If we are working in the probability space $(\Omega, \mathscr{G}, \mathbb{P})$ then a random variable is a measureable function $X: \Omega \rightarrow \mathbb{R}$ from the sample space to the real numbers.  Importantly, the random variable does not contain the information on its own distribution, so it is impossible to define the entropy of the distribution ---or any moments of the distribution--- purely as a function of the random variable.  It is possible to obtain the probability distribution from the random variable $X$ and the probability measure $\mathbb{P}$, so such a function can have both arguments, but it cannot be a function only of the random variable.
This means that whenever we use moment notation (where we express moments as a function of the random variable rather than the distribution), the functions must be implicitly conditional on the underlying probability measure $\mathbb{P}$ or the specific distribution of the random variable in question.  Sometimes we have a function where one or more arguments are only implicit, and do not appear in the function notation, and so it is certainly possible to write moments (and the entropy function) in this way.  It is in common usage in statistical problems.
This notation is a convenience because it allows you to relate the moments/entropy directly to random variables rather than their distributions.
A: $H$ is just a letter some smart guy use for Entropy. It is possible to use even $E$ for entropy but then there will be confusion with the expected value.
In here $X \sim \mathcal D$, means $\mathcal D$ is probability distribution of discrete random variable $X$.
$$
H[X] = - \sum_{x \sim \mathcal D,  \\ x \in R_X} P(x) \log_2 P(x)
$$
where:

*

*$P(x)$ is probability that $X=x$

*$x \sim \mathcal D$ ($x$ is also drawn from $\mathcal D$)

*$R_X$ is the support of $X$ (the set of all possible values $X$ can take)

A: A possible perspective is that it better accommodates the notation for conditional entropy (or mutual information): consider the common notation $H_P(X|Y)$, where $P_{XY}$ is understood to refer to the joint distribution on $XY$, versus alternatives one might consider such as e.g. $H(P_X|P_Y)$, which might mislead one into thinking it is a function of the marginal distributions $P_X$ and $P_Y$ alone.
(Of course, $H(P_X|P_Y)$ is not the only way one could try to denote this, but it seems very likely that any notation for conditional entropy will need to refer to $X$, $Y$ and the joint distribution $P_{XY}$ in some form - so you might as well put it together as $H_P(X|Y)$.)
Though in any case, you're not alone in suggesting the notation $H(P)$ when there is only one random variable - it's used in some places, but mostly when there is no danger of confusion along the lines of what's mentioned above.
