Skip to main content
edited body
Source Link

you probably refer to the asymptotic equipartition property? AEP Wikipedia $X_1,...,X_n$ have to be iid then, since $-\frac{1}{n}\log p(x_1,...,x_n)=-\frac{1}{n}\sum_i\log p(X_i)\rightarrow -E[\log p(X)] $$-\frac{1}{n}\log p(X_1,...,X_n)=-\frac{1}{n}\sum_i\log p(X_i)\rightarrow -E[\log p(X)] $ in probability, which is $H(X)$

you probably refer to the asymptotic equipartition property? AEP Wikipedia $X_1,...,X_n$ have to be iid then, since $-\frac{1}{n}\log p(x_1,...,x_n)=-\frac{1}{n}\sum_i\log p(X_i)\rightarrow -E[\log p(X)] $ in probability, which is $H(X)$

you probably refer to the asymptotic equipartition property? AEP Wikipedia $X_1,...,X_n$ have to be iid then, since $-\frac{1}{n}\log p(X_1,...,X_n)=-\frac{1}{n}\sum_i\log p(X_i)\rightarrow -E[\log p(X)] $ in probability, which is $H(X)$

Source Link

you probably refer to the asymptotic equipartition property? AEP Wikipedia $X_1,...,X_n$ have to be iid then, since $-\frac{1}{n}\log p(x_1,...,x_n)=-\frac{1}{n}\sum_i\log p(X_i)\rightarrow -E[\log p(X)] $ in probability, which is $H(X)$