Skip to main content
Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Code code:"if (foo != bar)"
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Saves in:saves
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with
Search options not deleted user 320402

A branch of mathematics/statistics used to determine the information carrying capacity of a channel, whether one that is used for communication or one that is defined in an abstract sense. Entropy is one of the measures by which information theorists can quantify the uncertainty involved in predicting a random variable.

0 votes

Joint entropy of multivariate normal distribution less than individual entropy under high co...

Recall that entropy is a measure for the uncertainty of a probability distribution. Distributions that are more uncertain and thus, observations from that particular distributions are harder to predic …
Sweetm's user avatar
  • 31
1 vote

Is it true that entropy estimation is meaningless if samples are not i.i.d.?

you probably refer to the asymptotic equipartition property? AEP Wikipedia $X_1,...,X_n$ have to be iid then, since $-\frac{1}{n}\log p(X_1,...,X_n)=-\frac{1}{n}\sum_i\log p(X_i)\rightarrow -E[\log p( …
Sweetm's user avatar
  • 31
1 vote

Mutual information relationship to copula entropy is borked?

I'll try to provide some intuition: As you already mentioned, differential entropy may become negative and that is exactly what happens here. Recap that differential entropy is some measure for the un …
Sweetm's user avatar
  • 31