# Questions tagged [maximum-entropy]

maximum entropy or maxent is a statistical principle derived from information theory. Distributions maximizing entropy (under some constraints) are thought to be "maximally uninformative" given the constraints. Maximum entropy can be used for multiple purposes, like choice of prior, choice of sampling model, or design of experiments.

162 questions
Filter by
Sorted by
Tagged with
1 vote
43 views

### How can we use shannon entropy to discriminate between two similar probability distribution function?

I studied two papers related to discriminating between two similar distributions using Shannon entropy. But both of them had different views. Can anyone explain what would be the basic flow of idea to ...
• 11
23 views

### Maximum entropy prior for binomial trial, is it 1/(2n+2) and this reasonable?

I am looking into what prior probability should be assigned to an event in a binomial trial that could occur but has not yet occurred after many trials. rephrased, what probability should be assigned ...
200 views

### Choosing "Target Entropy" for Soft-Actor-Critic (SAC) algorithm

I am quite familiar with Soft-Actor-Critic (SAC) and its many applications in continuous control RL environments. However, when implementing this algorithm in a practical setting, one thing that still ...
• 163
18 views

### Discrete Bayes Net learning under parameter constraints

What is some relevant research available on estimating the parameters of a Bayes Net (with known structure) when there are known constraints on conditional and marginal probabilities? For example, ...
• 1,428
21 views

### Differentiating entropy in Reinforcement Learning as Probabilistic Inference

I am studying the paper Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (https://arxiv.org/abs/1805.00909) and I do not understand how the author differentiate the ...
86 views

30 views

### How to statistically detect a treshold effect over a dependent variable measured repeated times on the same population

I want to identify the level of a predictive variable X (with Gaussian distribution) able to induce a reduction in a variable y (with Poisson distribution), that has been measured over the same ...
• 540
208 views

### Computing the gradient of the log-partition function in a linear-chain conditional random field (CRF) model

Query. When computing the gradient of the log-partition function for an exponential family distribution specified by the linear-chain conditional random field (CRF) model, will unary conditional ...
• 2,206
265 views

### Geometric distribution and entropy

According to wikipedia, among all discrete probability distributions supported on $\{1, 2, 3, ... \}$ with given expected value $\mu$, the geometric distribution X with parameter $p = \frac{1}{ \mu}$ ...
• 426
1 vote
56 views

### What is the maximum entropy joint Bernoulli distribution with fixed covariances and individual means?

We have Bernoulli variables $B_i$ with known means $E(B_i)$ and covariance matrix $\Sigma = (cov(B_i, B_j))$. What joint distribution would have the maximum entropy?
• 111
1 vote
55 views

### How to evaluate the likelihood of a conditional MAXENT estimation?

Suppose I have a random variable $Y$ (the outcome) and a set of random variables $\mathbf{X}$ (the input variables). I don't have access to observations of the joint distribution of $P(Y, \mathbf{X})$,...
• 260
70 views

• 63
38 views

• 1
21 views

### Maximum entropy prior for r.v. supported on real line with no other constraints?

What would be a suitable maximum entropy prior for a random variable supported on the real line with no other constraints (i.e. unknown mean, unknown variance, unknown bounds)? All kinds of answers (...
1 vote
152 views

### how to calculate entropy on matrix of words, topics

I have been digging in the concept of entropy for a while, now it comes to the implementation part I feel I am confused. Imagine that we have a matrix 20 * 3 standing for 20 words 3 topics (by 20 ...
• 228