Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Alright, I expect this is a silly question, but I don't actually know, so.

Suppose there is some random variable that's distributed on the reals, and all I know about the distribution is its mean $\mu$ and its variance $\sigma^2$. Maximum Entropy would have me say that I should choose the Normal distribution as the one that best fit my knowledge about it.

However, suppose I have an utility function about this variable that goes $U(x)=x^4$. Then, my expected utility for drawing from that variable's distribution is $EU(x) = \int\limits_{-\infty}^{+\infty}\!\!x^4\ p(x)\ \mathrm{d}x$. If I choose to use MaxEnt, then that's just $3\sigma^4$. However, if the "true" distribution actually followed by that random variable is, say, the Student's t with $\nu\leq 4$, then my Expected Utility would diverge to infinity.

If I treat the distribution as if I don't know what it is, then I'd have that $p(x)=\int\limits_{D\in\mathcal D}\!\!p(x|D)\ p(D)\ \mathrm{d}D$ where $\mathcal D$ is the space of all possible distributions. In that case, my expected utility is $EU(x) = \int\limits_{-\infty}^{+\infty}\!\int\limits_{D\in\mathcal D}\!\!x^4\ p(x|D)\ p(D)\ \mathrm{d}D\ \mathrm{d}x$ which I expect is not the same as the one I get by using the Normal except for very few priors over $\mathcal D$.

My questions then are: how do I deal with that? What prior distribution over $\mathcal D$ gives me the same result as MaxEnt would? In what sense is the Normal my best guess for my state of ignorance? How do I guarantee that this doesn't diverge? How does having the Normal as the "representative" of my subjective uncertainty take other possible distributions such as the Student's t into account?

Or, rephrasing this: I don't feel like the Gaussian actually represents my state-of-knowledge here, because nowhere in it is it represented that it's possible that the distribution has infinite fourth moment. In what sense is the Gaussian the best description of my state-of-knowledge? Is it, really? If not, what is?

share|improve this question

1 Answer 1

Maximum Entropy would have me say that I should choose the Normal distribution as the one that best fit my knowledge about it.

Maximum entropy isn't really about 'best fit', it's more about about choosing the distribution that maximizes ignorance (in a particular sense) conditional on some given information.

However, if the "true" distribution actually followed by that random variable is, say, the Student's t, then my Expected Utility would diverge to infinity.

This is untrue unless you also specify that the df, $\nu \leq 4$ . Finite 4th moments exist for all $\nu>4$.

But yes, a sufficiently-heavy-tailed distribution may not have a fourth moment, and hence if $U$ increases as a 4th power, it won't have an expected utility. If you believe expected utilities must exist then there's a problem with the model you're using, either in the distribution or the specification of the utility function.

Your other questions seem to be getting at things that relate to your choice of models, and ultimately your own preferences. Unless you ask more specific questions, I am not sure that we can say much.

share|improve this answer
    
There are two mathematical objects: the one that represents my state of knowledge about the distribution, and the actual distribution itself. My utility function has a term for drawing elements from the actual distribution, which is unknown by me, and that term goes as the fourth power of the element drawn. If the Gaussian is the one that best represents my own state of knowledge, isn't it completely ignoring the fact that I know that the actual distribution could be a Student's t with $\nu\leq 4$? –  Pedro Carvalho Jul 30 at 18:04
    
If I give that distribution any probability of being the actual one, then it completely swamps my utility function and I should draw things from that distributed thing forever. So what I'm asking is, what's the actual model that best represents my own state of knowledge? –  Pedro Carvalho Jul 30 at 18:04
    
I don't understand what "I should draw things from that distributed thing forever" is getting at. The rest of it is simply saying "it's possible to assume a utility and a distribution that make the expected utility infinite, or even not exist at all". This is the case, but why is that surprising? There's nothing to do about it; it's not finite in that instance. Why should expected utility exist? –  Glen_b Jul 30 at 18:07
    
It could mean whatever, it depends on the variable. For instance, suppose I'm an AI and there's a box with a button, and when I press that button, some real number is randomly chosen according to a distribution of which I only know mean and variance. Suppose also that this box is directly hooked to my pleasure center such that I get hedons/utilons as the fourth power of the chosen number. Then I will be stuck in an infinite loop of pressing that button forever no matter what else happens in the world; I'll have wireheaded myself by accident. Why wouldn't expected utility exist? –  Pedro Carvalho Jul 30 at 22:07
    
"Why wouldn't expected utility exist?" -- It seems like you're engaged in the fallacy of argument from ignorance. If you can't produce a reason to think it should, there's nothing to respond to, particularly since it's already not to be finite, nor even to exist at all in particular instances. I don't understand the difficulty - if you posit conditions where it isn't finite, it isn't finite, end of story. Why would there be anything to do about that? –  Glen_b Jul 30 at 22:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.