# $\mathbb{E}(X^2+Y^2)=-2\rho$? $(X,Y)$ is from standard bivariate normal distribution and $Cov(X,Y)=\rho$

How to use log partition function to derive $$\mathbb{E}(X^2+Y^2)$$, where $$(X,Y)$$ is from standard bivariate normal distribution? By standard bivariate normal I mean $$\mu_x=\mu_y=0$$ and $$\sigma^2_X=\sigma^2_Y=1$$ and $$\sigma_{XY}=\rho$$. The answer is clearly $$\mathbb{E}(X^2+Y^2)=2$$ but I don't know how to use the log partition function to calculate it. My calculation gave me $$\mathbb{E}(X^2+Y^2)=-2\rho^2$$.

Here is what I've done.

The p.d.f of $$(X,Y)$$ is $$f_{X,Y}(x,y)=\frac{1}{2\pi\sqrt{1-\rho^2}}\exp\left\{-\frac{1}{2(1-\rho^2)}\left(x^2+y^2-2\rho xy\right)\right\}$$ Thus we can write $$f_{X,Y}(x,y)$$ in a exponential family form:

$$f_{X,Y}(x,y)=\exp\left\{(x^2+y^2,xy)\cdot(-\frac{1}{2(1-\rho^2)},\frac{\rho}{1-\rho^2})-\frac{1}{2}\log{(1-\rho^2)}\right\},$$ from where we can see that the sufficient statistic $$T=(X^2+Y^2, XY)'$$ and the natural parameter $$\eta = (\eta_1,\eta_2)'= \left(-\frac{1}{2(1-\rho^2)}, \frac{\rho}{1-\rho^2}\right)'$$ and the log-partition function is $$A(\eta)=\frac{1}{2}\log{(1-\rho^2)}=\frac{1}{2}\log{(1-\frac{\eta_2^2}{4\eta_1^2})}$$

Since we can derive the expected value of $$T$$ using $$\mathbb{E}(T)=\nabla_\eta A(\eta)$$ then in our case we have $$\nabla_\eta A(\eta)=\left(\frac{\eta_2^2}{4\eta_1^3-\eta_2^2\eta_1}, \frac{\eta_2}{\eta_2^2-4\eta_1^2}\right)'$$

I am pretty confidence about the above derivation as I used an online tool to calculate it.

However, if I plug $$\eta_1= -\frac{1}{2(1-\rho^2)}$$ and $$\eta_2= \frac{\rho}{1-\rho^2}$$, what I got is $$\nabla_\eta A(\eta)|_{\eta=\eta(\rho)}=\left(-2\rho^2,-\rho\right)'=\mathbb{E}(T_1,T_2)=\mathbb{E}(X^2+Y^2,XY)$$

But this is apparently not correct as we all know that $$\mathbb{E}(XY)=\rho$$ and $$\mathbb{E}(X^2+Y^2)=2$$.

Where I did wrong?

This is a curved exponential family, which is a subset of the bigger full-rank family $$\text{density} \propto \exp\big( \eta_1(x^2 + y^2) + \eta_2 xy - A(\eta)\big).$$ Consider computing the log-partition function for this larger family. You can write down the density for $$(X,Y)$$ assuming that $$\text{var}(X) = \text{var}(Y) = \sigma^2$$ and $$\text{cor}(X,Y) = \rho$$, that is, correlation coefficient $$= \rho$$. By comparing the density to the above form you can figure out $$\eta_1$$ and $$\eta_2$$ and $$A(\eta)$$.

What you are trying to do is to recover $$A(\eta)$$ from only knowing $$A(\tilde\eta(\rho))$$ for some function $$\widetilde \eta: \mathbb R \to \mathbb R^2$$. This in general is not doable. There are many functions $$A: \mathbb R^2 \to \mathbb R$$ such that $$A(\widetilde \eta(\rho)) = \frac12 \log(1-\rho^2)$$.

Don't read the rest and try it out yourself.

The correct log-partition function for the full family turns out to be something like this: $$A(\eta) = -\frac12 \log \Big( \frac14 \big(\eta_1^2 - \frac14 \eta_2^2\big) \Big)$$ with $$\sigma^2 = -(\eta_1/2) /(\eta_1^2 - \eta_2^2/4)$$.

• Thank you. I see your point. Is there any way of not by introducing a new parameter, e.g. $\sigma^2$? Is there any general way of getting the correct log-partition function for the curved exponential family?
– Tan
Jan 19 at 0:37
• You can try to integrate the expression for the full-rank model. BTW, there is no unique curved family. You can get different curved families from the full-rank model. Pick any reasonable function $g : \mathbb R \to \mathbb R^2$ and use the reparametrization, $\eta = g(t)$. The log-partition function of the resulting model does not have enough information (as far as I understand) to reconstruct the log-partition function of the original. Jan 19 at 0:47
• Exactly. I agree with you that different $g$ can give different log-partition function. However, integrating the expression for the full-rank model, e.g. $A(\eta) = \int e^{T'\eta}\mu(dx),\eta\in\Xi$ is not easy sometimes. Maybe a clever way is to notice the corresponding full-rank family with parameter $\theta$, where $\eta = \eta(\theta)$, e.g. noticing $\sigma^2$ in this case.
– Tan
Jan 19 at 0:56