$\mathbb{E}(X^2+Y^2)=-2\rho$? $(X,Y)$ is from standard bivariate normal distribution and $Cov(X,Y)=\rho$

Question

How to use log partition function to derive $\mathbb{E}(X^2+Y^2)$, where $(X,Y)$ is from standard bivariate normal distribution? By standard bivariate normal I mean $\mu_x=\mu_y=0$ and $\sigma^2_X=\sigma^2_Y=1$ and $\sigma_{XY}=\rho$. The answer is clearly $\mathbb{E}(X^2+Y^2)=2$ but I don't know how to use the log partition function to calculate it. My calculation gave me $\mathbb{E}(X^2+Y^2)=-2\rho^2$.

Here is what I've done.

The p.d.f of $(X,Y)$ is $$f_{X,Y}(x,y)=\frac{1}{2\pi\sqrt{1-\rho^2}}\exp\left\{-\frac{1}{2(1-\rho^2)}\left(x^2+y^2-2\rho xy\right)\right\}$$ Thus we can write $f_{X,Y}(x,y)$ in a exponential family form:

$$f_{X,Y}(x,y)=\exp\left\{(x^2+y^2,xy)\cdot(-\frac{1}{2(1-\rho^2)},\frac{\rho}{1-\rho^2})-\frac{1}{2}\log{(1-\rho^2)}\right\},$$ from where we can see that the sufficient statistic $$T=(X^2+Y^2, XY)'$$ and the natural parameter $$\eta = (\eta_1,\eta_2)'= \left(-\frac{1}{2(1-\rho^2)}, \frac{\rho}{1-\rho^2}\right)'$$ and the log-partition function is $$A(\eta)=\frac{1}{2}\log{(1-\rho^2)}=\frac{1}{2}\log{(1-\frac{\eta_2^2}{4\eta_1^2})}$$

Since we can derive the expected value of $T$ using $$\mathbb{E}(T)=\nabla_\eta A(\eta)$$ then in our case we have $$\nabla_\eta A(\eta)=\left(\frac{\eta_2^2}{4\eta_1^3-\eta_2^2\eta_1}, \frac{\eta_2}{\eta_2^2-4\eta_1^2}\right)'$$

I am pretty confidence about the above derivation as I used an online tool to calculate it.

However, if I plug $\eta_1= -\frac{1}{2(1-\rho^2)}$ and $\eta_2= \frac{\rho}{1-\rho^2}$, what I got is $$\nabla_\eta A(\eta)|_{\eta=\eta(\rho)}=\left(-2\rho^2,-\rho\right)'=\mathbb{E}(T_1,T_2)=\mathbb{E}(X^2+Y^2,XY)$$

But this is apparently not correct as we all know that $\mathbb{E}(XY)=\rho$ and $\mathbb{E}(X^2+Y^2)=2$.

Where I did wrong?

Answer 1

This is a curved exponential family, which is a subset of the bigger full-rank family $$ \text{density} \propto \exp\big( \eta_1(x^2 + y^2) + \eta_2 xy - A(\eta)\big). $$ Consider computing the log-partition function for this larger family. You can write down the density for $(X,Y)$ assuming that $\text{var}(X) = \text{var}(Y) = \sigma^2$ and $\text{cor}(X,Y) = \rho$, that is, correlation coefficient $ = \rho$. By comparing the density to the above form you can figure out $\eta_1$ and $\eta_2$ and $A(\eta)$.

What you are trying to do is to recover $A(\eta)$ from only knowing $A(\tilde\eta(\rho))$ for some function $\widetilde \eta: \mathbb R \to \mathbb R^2$. This in general is not doable. There are many functions $A: \mathbb R^2 \to \mathbb R$ such that $A(\widetilde \eta(\rho)) = \frac12 \log(1-\rho^2)$.

Don't read the rest and try it out yourself.

---

The correct log-partition function for the full family turns out to be something like this: $$ A(\eta) = -\frac12 \log \Big( \frac14 \big(\eta_1^2 - \frac14 \eta_2^2\big) \Big) $$ with $\sigma^2 = -(\eta_1/2) /(\eta_1^2 - \eta_2^2/4)$.