MaxEnt model vs cross entropy loss Pardon my ignorance. I am still learning. 
We try to minimize the cross-entropy loss for best results.
However, why should the entropy be high for a MaxEnt model for the model to be good?
My understanding is that lower entropy models are good.
Please give me an intuitive explanation of the MaxEnt models.
Thanks in advance.
 A: The Maximum-Entropy formalism can be tedious to understand. Obviously low entropy is a good thing. Entropy is used to quantify the uncertainty (synonymous with "randomness" or "disorder", as you will see throughout Max-Ent discussions) of a system of interest, and your question "why should the entropy be high for a MaxEnt model for the model to be good?" is perfectly reasonable and it is counter-intuitive why we would want a model to maximize its uncertainty. 
The essential thing to understand about Maximum-Entropy is it makes no assumptions. Every Maximum-Entropy model starts with two fundamentals of both entropy and probability,
$$H = - \sum_i^N p_i \log p_i \tag{1}$$
and,
$$\sum_i^N p_i = 1\tag{2}$$
equation (1) is obviously the entropy equation and equation (2) simply says that all of our probabilities add to one. We have not used any intuition, made any guesses or assumptions to further describe any system except that fundamentally all systems have an entropy and a probability distribution.
Let us take a system described only by the two equations above as an example, i.e. suppose we have a system that we have no information about, but we know that all systems must follow equations (1) and (2). The Maximum-Entropy model produces the intuitive result that if no additional is known, the probability should be distributed uniformly, $p_i = 1/N$. 
If we add another constraint, that the mean value of system is explicitly given, 
$$\sum_i^N x_i p_i = \mu \tag{3}$$
Maximum-Entropy will again, take only the additional information that is given to produce a probability distribution that is the most uncertain constrained on the information provided. Following Max-Ent with the mean constraint produces again another intuitive result of an exponential distribution, $p_i \sim e^{- \frac{x_i}{\mu}}$. Knowing nothing else about the system, the probability density is higher closer to the mean and lower farther away from the mean. 
The 1st example has a higher entropy (uncertainty), all things held equal, compared to the 2nd but only because Max-Ent was explicitly given more information about the system. 
Maximum-Entropy the model that knows nothing but fundamental probability and entropy, and the information explicitly described by the system.
