I have what I hope should be a relatively quick and easy to answer question.
Would it ever make sense to replace the probabilities by an objective function within the relative entropy formulation?
I imagine a form of robust decision making under which a decision maker wishes to choose some value $s$ that maximises some function $g(x,s)$ say, but is uncertain as to the underlying distribution of $X$ except that is has distribution $F \in \mathcal{D}$ say, where $\mathcal{D}$ may be a constrained set of distribution functions (e.g. moment constraints). They will want to choose the value of $s$ that is most 'robust' against the distributional uncertainty relative to the true optimum, call it $g(x,t)$. This may look something like:
$\max_{s\geq 0}\min_{F\in \mathcal{D}}\min_{t\geq 0}\int_{\Theta_X} f(x)\log\frac{g(x,s)}{g(x,t)}\,\mathrm{d}x $
In words the problem itself might be though of as the average of the logarithmic relative regret of choosing $s$ over $t$.
If anyone has any thoughts on why this might be a bad idea, or problems that I would likely run in to then I would be grateful to hear from you.
My main question comes really is whether this would make sense?
