Infinite fourth moment and maximum entropy Alright, I expect this is a silly question, but I don't actually know, so.
Suppose there is some random variable that's distributed on the reals, and all I know about the distribution is its mean $\mu$ and its variance $\sigma^2$. Maximum Entropy would have me say that I should choose the Normal distribution as the one that best fit my knowledge about it.
However, suppose I have an utility function about this variable that goes $U(x)=x^4$. Then, my expected utility for drawing from that variable's distribution is $EU(x) = \int\limits_{-\infty}^{+\infty}\!\!x^4\ p(x)\ \mathrm{d}x$. If I choose to use MaxEnt, then that's just $3\sigma^4$. However, if the "true" distribution actually followed by that random variable is, say, the Student's t with $\nu\leq 4$, then my Expected Utility would diverge to infinity.
If I treat the distribution as if I don't know what it is, then I'd have that $p(x)=\int\limits_{D\in\mathcal D}\!\!p(x|D)\ p(D)\ \mathrm{d}D$ where $\mathcal D$ is the space of all possible distributions. In that case, my expected utility is $EU(x) = \int\limits_{-\infty}^{+\infty}\!\int\limits_{D\in\mathcal D}\!\!x^4\ p(x|D)\ p(D)\ \mathrm{d}D\ \mathrm{d}x$ which I expect is not the same as the one I get by using the Normal except for very few priors over $\mathcal D$.
My questions then are: how do I deal with that? What prior distribution over $\mathcal D$ gives me the same result as MaxEnt would? In what sense is the Normal my best guess for my state of ignorance? How do I guarantee that this doesn't diverge? How does having the Normal as the "representative" of my subjective uncertainty take other possible distributions such as the Student's t into account?
Or, rephrasing this: I don't feel like the Gaussian actually represents my state-of-knowledge here, because nowhere in it is it represented that it's possible that the distribution has infinite fourth moment. In what sense is the Gaussian the best description of my state-of-knowledge? Is it, really? If not, what is?
 A: 
Maximum Entropy would have me say that I should choose the Normal distribution as the one that best fit my knowledge about it.

Maximum entropy isn't really about 'best fit', it's more about about choosing the distribution that maximizes ignorance (in a particular sense) conditional on some given information. 

However, if the "true" distribution actually followed by that random variable is, say, the Student's t,  then my Expected Utility would diverge to infinity.

This is untrue unless you also specify that the df, $\nu \leq 4$ . Finite 4th moments exist for all $\nu>4$.
But yes, a sufficiently-heavy-tailed distribution may not have a fourth moment, and hence if $U$ increases as a 4th power, it won't have an expected utility. If you believe expected utilities must exist then there's a problem with the model you're using, either in the distribution or the specification of the utility function.
Your other questions seem to be getting at things that relate to your choice of models, and ultimately your own preferences. Unless you ask more specific questions, I am not sure that we can say much.
