# Questions tagged [maximum-entropy]

maximum entropy or maxent is a statistical principle derived from information theory. Distributions maximizing entropy (under some constraints) are thought to be "maximally uninformative" given the constraints. Maximum entropy can be used for multiple purposes, like choice of prior, choice of sampling model, or design of experiments.

162 questions
Filter by
Sorted by
Tagged with
0answers
113 views

### Computing the gradient of the log-partition function in a linear-chain conditional random field (CRF) model

Query. When computing the gradient of the log-partition function for an exponential family distribution specified by the linear-chain conditional random field (CRF) model, will unary conditional ...
0answers
14 views

### Applying principle of maximum entropy with neural network

I am looking for resources regarding how I can maximize entropy to a model trained on artificial neural networks. While I have been able to find papers for classification tasks, like this one, I also ...
0answers
201 views

### Geometric distribution and entropy

According to wikipedia, among all discrete probability distributions supported on $\{1, 2, 3, ... \}$ with given expected value $\mu$, the geometric distribution X with parameter $p = \frac{1}{ \mu}$ ...
0answers
10 views

### What is the maximum entropy distribution given *conditional* means and MADs?

I know the maximum entropy distribution given the mean and MAD (Mean Absolute Difference) around the mean (it's the Laplace distribution, a proof here for example). I also know the maximum entropy ...
0answers
31 views

### What is the maximum entropy joint Bernoulli distribution with fixed covariances and individual means?

We have Bernoulli variables $B_i$ with known means $E(B_i)$ and covariance matrix $\Sigma = (cov(B_i, B_j))$. What joint distribution would have the maximum entropy?
0answers
40 views

### How to evaluate the likelihood of a conditional MAXENT estimation?

Suppose I have a random variable $Y$ (the outcome) and a set of random variables $\mathbf{X}$ (the input variables). I don't have access to observations of the joint distribution of $P(Y, \mathbf{X})$,...
0answers
50 views

1answer
26 views

0answers
19 views

### Maximum entropy prior for r.v. supported on real line with no other constraints?

What would be a suitable maximum entropy prior for a random variable supported on the real line with no other constraints (i.e. unknown mean, unknown variance, unknown bounds)? All kinds of answers (...
1answer
94 views

### how to calculate entropy on matrix of words, topics

I have been digging in the concept of entropy for a while, now it comes to the implementation part I feel I am confused. Imagine that we have a matrix 20 * 3 standing for 20 words 3 topics (by 20 ...
0answers
39 views