1
$\begingroup$

Let $\Phi$ and $\phi$ respectively be the cumulative distribution function and probability density function of a standard normal distribution. $\beta$ is a $d \times 1$ vector which follows a multivariate normal distribution $\mathcal{N}\left(\mu, \Sigma\right)$. How can I compute the following expectation: $$ \mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right] $$

I wonder if this is the same as $\log \Phi\left(x^{T}\mu\right)$.

$\endgroup$
5
  • $\begingroup$ Do you know how to do it when it is $\mu$ instead of $\beta$? $\endgroup$ Feb 11, 2016 at 7:06
  • $\begingroup$ @RustyStatistician There would be no point in taking the expectation if there was no $\beta$. My question was if I could just plug in the mean vector of $\beta$ even though it's inside both the log and the cdf... (I don't think it's possible since those functions are not linear.) $\endgroup$
    – Daeyoung
    Feb 11, 2016 at 7:09
  • $\begingroup$ I was confused because you said "I wonder if this is the same as $\log \Phi\left(x^{T}\mu\right)$" $\endgroup$ Feb 11, 2016 at 7:11
  • $\begingroup$ This may help stats.stackexchange.com/questions/57715/… $\endgroup$ Feb 11, 2016 at 7:19
  • 1
    $\begingroup$ Recall $g(E(X))\neq E(g(X))$ for nonlinear functions $g$, see en.wikipedia.org/wiki/Jensen%27s_inequality $\endgroup$ Feb 11, 2016 at 7:26

1 Answer 1

4
$\begingroup$

It should be noted that, in general, for any non-linear function $g$ of a random variable $X$, $ \mathbb{E}[g(X)] \neq g(\mathbb{E}[X]) $ . You can see this by expanding out the integrals; $$ \mathbb{E}[g(X)] = \int_X g(x)f(x)dx;\;\;\;\;g(\mathbb{E}[X])=g\bigg(\int_X xf(x)dx\bigg) $$ You could see how the two would only be equal in the case $g$ was linear.

So given the above; $$\mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right] \neq \log \Phi\left(x^{T}\mu\right)$$

You can estimate $\mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right] $ via Monte Carlo

$$ \mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right]=\int_\beta \log \Phi \left(x^{T}\beta\right)f(\beta|\mu, \Sigma)d\beta \approx \frac{1}{G}\sum_{g=1}^G \log \Phi \left(x^{T}\beta^{(g)}\right) $$ where $\beta^{(1)},\beta^{(2)},...,\beta^{(G)} \sim \mathcal{N}(\mu,\Sigma)$.

If the dimension of $\beta$ is large, the above approximation may not work efficiently (it will take a very large $G$ for it to get close). Otherwise, this is a pretty standard and well accepted approach.

I am not aware of a closed form solution to the above integral. If another user does derive an analytical solution you should use that instead of the the Monte Carlo approximation above. There may also be ways to improve the above estimate with importance sampling.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.