# Do IID experiments maximize Fisher information?

Let $x$ and $y$ be random variables whose probabilities depend on an unknown parameter $\theta$. I am specifically interested in the case that both $x$ and $y$ are Bernoulli, but the question below can also be asked in general. Assuming sufficient regularity, the Fisher information of a set $X$ of random variables about $\theta$ is given by the expression:

$\mathcal{I}_X(\theta)= -E\left[\left.\frac{\partial^2}{\partial\theta^2}\log f(X;\theta)\,\right|\theta\right]\enspace ,$

where $f(X;\theta)$ is the probability density of $X$ conditional on $\theta$.

If $x$ and $y$ are independent, then:

$\mathcal{I}_{x,y}(\theta)= \mathcal{I}_{x}(\theta)+\mathcal{I}_{y}(\theta)\enspace .$

Question: Is it always true (even when $x$ and $y$ are dependent) that $\mathcal{I}_{x,y}(\theta)\leq \mathcal{I}_{x}(\theta)+\mathcal{I}_{y}(\theta)$?

My intuition from Shannon information tells me that surely this is so- if you have a coin, and you toss it once, your optimal strategy for the second toss is surely just to toss it again. But I've been unable to find a proof for this, and the analysis gets very complicated very fast when I try to prove this "subadditivity" property analytically.

Question: Do you have a reference for this property?

• Thanks! Is this the correct subadditivity property though? It looks to me like subadditivity with respect to something else- namely $\mathcal{I}_{T(x,y)}\leq \mathcal{I}_{x,y}$ for any function T, where x,y are iid (equation 1.5 in Stam). Have I missed something? Commented Jun 11, 2015 at 21:42