# Tag Info

### Statistical interpretation of Maximum Entropy Distribution

This isn't really my field, so some musings: I will start with the concept of surprise. What does it mean to be surprised? Usually, it means that something happened that was not expected to ...
• 66.4k

### Why is Entropy maximised when the probability distribution is uniform?

Entropy in physics and information theory are not unrelated. They're more different than the name suggests, yet there's clearly a link between. The purpose of entropy metric is to measure the amount ...
• 56.8k
Accepted

### Prove that the maximum entropy distribution with a fixed covariance matrix is a Gaussian

The starred step is valid because (a) $p$ and $q$ have the same zeroth and second moments and (b) $\log(p)$ is a polynomial function of the components of $\mathbf{x}$ whose terms have total degrees $0$...
• 294k

### Is differential entropy always less than infinity?

I thought about this question some more and managed to find a counter-example, thanks also to the Piotr's comments above. The answer to the first question is no - the differential entropy of a ...
• 806

Accepted

### Is it possible to use entropy maximisation to get around collinearity in linear regression?

Principal components regression is one well respected way to handle collinear predictors, and my sense is that it has a relation to entropy maximization. I don't have time right now to think this ...
• 66.5k

### Justification for invoking Maximum Entropy

First of all, I tried to comment on the question but I couldn't because I didn't have (still don't) 50 reputation, so I'm posting my opinion as an answer despite knowing it is not a complete answer to ...
• 304

### Infinite fourth moment and maximum entropy

Maximum Entropy would have me say that I should choose the Normal distribution as the one that best fit my knowledge about it. Maximum entropy isn't really about 'best fit', it's more about about ...
• 263k

Only top scored, non community-wiki answers of a minimum length are eligible