Suppose I know the value of n specific quantiles from a large sample. What is the maximum entropy distribution over the real line given those values?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ Have you tried formulating and solving this as an optimization problem of maximizing entropy subject to equality constraints on the n known quantiles? I think you should be able to get the idea of things by Googling and looking at similar problems. $\endgroup$– Mark L. StoneCommented Aug 11, 2016 at 1:47
-
$\begingroup$ I'll poke about and see what I can find. But if g1ul10's response, below, is correct -- and I think it is -- then that approach will not work without further constraints. $\endgroup$– andrewHCommented Aug 18, 2016 at 0:23
-
$\begingroup$ Here is a similar problem with answers: stats.stackexchange.com/questions/361476/… $\endgroup$– kjetil b halvorsen ♦Commented Jul 1, 2023 at 1:08
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Maximum entropy problems do not always admit a solution. The generic expression for the maximum entropy density $f(x)$ given a set of integral constraints \begin{equation} \int dx \, h_i(x) \, f(x) = c_i \end{equation} with $i=1\ldots N$ is \begin{equation} f(x) = e^{\mu + \sum_{i=1}^N \lambda_i h_i(x)} \;. \end{equation} The values of the parameter $\mu$ and $\lambda_i$ have to be found by imposing the fulfillment of the constraints and the fact the $f(x)$ integrates to one, that is it is a proper density. I left the boundaries of the integral in the constraints unspecified on purpose. The reason will become clear in a moment.
The quantile constraints $F(x_i)=q_i$ where $F$ is the distribution function (the integral of the $f(x)$), translate in having $h_i(x)=1-\theta_{x_i}(x)$ and $c_i=q_i$, where $\theta_z(x)$ is the Heaviside theta function, which is equal to $1$ if $x>z$ and zero otherwise. The problem is that now the expression for $f(x)$ given above is not an (improper) integrable function on the real line. Summarizing, your problem does not have a solution. If however you add further constraints, like a finite support or some specified moment, than the problem might become solvable.
-
$\begingroup$ I don't have moments, etc., because high and low values are censored. Would it be sufficient if I had a candidate functional form? $\endgroup$– andrewHCommented Aug 18, 2016 at 0:20
-
$\begingroup$ So, that follow-up question was meaningless. The maximum entropy distribution would be the candidate distribution if there was one. Sorry. Here is a question that makes more sense: Would the observed quantiles suffice for a strictly positive distribution? $\endgroup$– andrewHCommented Aug 26, 2016 at 20:19
-
$\begingroup$ @andrewH No. The solution would have a constant behavior for large $x$ and the integral on the positive half-line would not exist. You could find a solution if you consider a finite domain, like $[0,1]$. In this case, however, the solution will be rather trivial, with a step-wise shape. $\endgroup$– g1ul10Commented Aug 29, 2016 at 8:13
-
$\begingroup$ @g1ul10 can the quantiles be used to estimate the information entropy of the sample? $\endgroup$– KuzekoCommented Dec 11, 2018 at 22:52