4
$\begingroup$

I want to have a PDF that maximizes entropy for a given mode value. I searched in Maximum entropy probability distribution but here we have maximization done over a certain moment constraint.

$\endgroup$

1 Answer 1

3
$\begingroup$

There is no clear meaning to this problem: when the mode is defined as the maximum of the density$$\hat\theta=\arg\max_{\theta\in\Theta} f(\theta)$$the value of $\hat\theta$ depends on the choice of the dominating measure $\lambda$, meaning that if one switches from one dominating measure $\lambda$ to another one $\lambda'$ that is absolutely continuous wrt the first one, the density is modified from $\frac{\text{d}P}{\text{d}\lambda}$ to $\frac{\text{d}P}{\text{d}\lambda'}$, hence opening the possibility of shifting the mode. The entropy $$\mathfrak{H}=-\mathbb{E}^P\left[\log \frac{\text{d}P}{\text{d}\lambda}(X)\right]$$ is also dependent on the dominating measure: $$\mathbb{E}^P\left[\log \frac{\text{d}P(X)}{\text{d}\lambda}\right] = \mathbb{E}^P\left[\log \frac{\text{d}P}{\text{d}\lambda'}(X)\right] +\underbrace{\mathbb{E}^P\left[\log \frac{\text{d}\lambda'}{\text{d}\lambda}(X)\right]}_{\text{depending on $P$}\\\text{hence involved in }\\\text{the maximisation}}$$

$\endgroup$
1
  • $\begingroup$ does mean and variance of a pdf remains invariant under reparametrization and therefore we have PDF that maximizes entropy over given mean and variance (Gaussian)? $\endgroup$
    – Userhanu
    Commented Apr 3, 2019 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.