I want to have a PDF that maximizes entropy for a given mode value. I searched in Maximum entropy probability distribution but here we have maximization done over a certain moment constraint.
1 Answer
There is no clear meaning to this problem: when the mode is defined as the maximum of the density$$\hat\theta=\arg\max_{\theta\in\Theta} f(\theta)$$the value of $\hat\theta$ depends on the choice of the dominating measure $\lambda$, meaning that if one switches from one dominating measure $\lambda$ to another one $\lambda'$ that is absolutely continuous wrt the first one, the density is modified from $\frac{\text{d}P}{\text{d}\lambda}$ to $\frac{\text{d}P}{\text{d}\lambda'}$, hence opening the possibility of shifting the mode. The entropy $$\mathfrak{H}=-\mathbb{E}^P\left[\log \frac{\text{d}P}{\text{d}\lambda}(X)\right]$$ is also dependent on the dominating measure: $$\mathbb{E}^P\left[\log \frac{\text{d}P(X)}{\text{d}\lambda}\right] = \mathbb{E}^P\left[\log \frac{\text{d}P}{\text{d}\lambda'}(X)\right] +\underbrace{\mathbb{E}^P\left[\log \frac{\text{d}\lambda'}{\text{d}\lambda}(X)\right]}_{\text{depending on $P$}\\\text{hence involved in }\\\text{the maximisation}}$$
-
$\begingroup$ does mean and variance of a pdf remains invariant under reparametrization and therefore we have PDF that maximizes entropy over given mean and variance (Gaussian)? $\endgroup$– UserhanuCommented Apr 3, 2019 at 6:29