Let x and y be random variables whose probabilities depend on an unknown parameter θ. I am specifically interested in the case that both x and y are Bernoulli, but the question below can also be asked in general. Assuming sufficient regularity, the Fisher information of a set X of random variables about θ is given by the expression:
IX(θ)=−E[∂2∂θ2logf(X;θ)|θ],
where f(X;θ) is the probability density of X conditional on θ.
If x and y are independent, then:
Ix,y(θ)=Ix(θ)+Iy(θ).
Question: Is it always true (even when x and y are dependent) that Ix,y(θ)≤Ix(θ)+Iy(θ)?
My intuition from Shannon information tells me that surely this is so- if you have a coin, and you toss it once, your optimal strategy for the second toss is surely just to toss it again. But I've been unable to find a proof for this, and the analysis gets very complicated very fast when I try to prove this "subadditivity" property analytically.
Question: Do you have a reference for this property?