Assume that we have two multivariate normal distributions $\mathcal{N}_1 = \mathcal{N}(\mu_1, \Sigma_1)$ and $\mathcal{N}_2 = \mathcal{N}(\mu_2, \Sigma_2)$. We do these two steps:

  1. Pick a point, say $x$, from $\mathcal{N}_1$.
  2. compute $\mathcal{N}_2(x)$ (probability density of x in $\mathcal{N}_2$).

What is the best correct statistical notation to show expected value for $\mathcal{N}_2(x)$?

The only notations that come to my mind are $E_{x\sim\mathcal{N}_1} \left( \mathcal{N}_2(x) \right)$ and $E_{x\sim\mathcal{N}(\mu_1, \Sigma_1)} \left( p_{\mathcal{N}(\mu_2, \Sigma_2)}(x) \right)$

NOTE 1: Instead of $\mathcal{N}_1, \mathcal{N}_2$ (numerical subscripts), I have to show the actual parameters $ \left( \mu_1, \Sigma_1, \mu_2, \Sigma_2 \right)$ in the formula.

NOTE 2: I already know the solution (It is solved here). I am just thinking about the correct notation.

  • 1
    You are confused, $\mathcal{N}_2(x)$ is not the probability of $x$ in ..., it is the probability density of $\mathcal{N}_2(x)$ in .... Your first proposed notation could be OK, given that you expålain it! – kjetil b halvorsen Aug 19 '14 at 9:01
  • @kjetilbhalvorsen: Thanks, I corrected the post. However, I think the suggested formula $(E_{x\sim\mathcal{N}(\mu_1, \Sigma_1)} \left( p_{\mathcal{N}(\mu_2, \Sigma_2)}(x) \right))$ is not straightforward for typesetting and reading, specially the $\sim$ mark. Is there any alternative form? E.g. $E(x \: ;\: \cdots|\: \cdots \:)$ ??? – Ali Aug 19 '14 at 9:39

Although the Normal distribution has a special status among distributions, I don't see the need to create a special notation here, it is unnecessary notational burden.
The way I see it, your problem starts from the fact that you use the symbol of the distribution to denote also the random vector itself. This is usual (especially in asymptotic results), but strictly speaking is wrong, because the random vector is a separate entity from the distribution that it follows. When we use the same symbol for the two, we conflate things badly.

So I would write "$\mathbf X_1 \sim \mathcal{N}(\mu_1, \Sigma_1)$" instead of "$\mathcal{N}_1 = \mathcal{N}(\mu_1, \Sigma_1)$" and analogously for the 2nd random vector. Then I would need only to additionally write one more piece of very standard notation: "Let $f_{X_1}(\mathbf x_1)$ be the density function of $\mathbf X_1$ and $f_{X_2}(\mathbf x_2)$ be the density function of $\mathbf X_2$" to immediately be able to write the expected value you want as

$$E[f_{X_2}(\mathbf X_1)]$$

The above conveys clearly that the "source of randomness" here is $\mathbf X_1$ only, (and so that the expected value should be taken with respect to the joint density of $\mathbf X_1$), since $f_{X_2}(\cdot)$ is used only to indicate the functional form of this function of $\mathbf X_1$. The fact that $f_{X_2}(\cdot)$ can operate also as a density function has no stochastic consequences here. In this notation, what distributions are involved is not apparent, but I don't see this as a problem -after all, this expression will come after a few lines of declaring some basic assumptions.

This case helps also to illustrate the usefulness of using $\mathbf X_1, \mathbf x_1$ to denote the random vector and a realization of it respectively. Also it shows that it is useful to index different random vectors ($\mathbf X_1$, $\mathbf X_2$, or use different symbols all together ($\mathbf X$, $\mathbf Y$), although admittedly, indexing the same letter helps keep in the picture that these random vectors may have identical distribution, or at least distributions that belong to the same family.

Finally, note that we do not have some conditional expected value here, since $E[f_{X_2}(\mathbf X_1) \mid \mathbf X_1 =\mathbf x_1] =f_{X_2}(\mathbf x_1)$ (no uncertainty remains), and so the "$\mid$" symbol should be avoided.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.