Let $\Phi$ and $\phi$ respectively be the cumulative distribution function and probability density function of a standard normal distribution. $\beta$ is a $d \times 1$ vector which follows a multivariate normal distribution $\mathcal{N}\left(\mu, \Sigma\right)$. How can I compute the following expectation: $$ \mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right] $$

I wonder if this is the same as $\log \Phi\left(x^{T}\mu\right)$.

  • $\begingroup$ Do you know how to do it when it is $\mu$ instead of $\beta$? $\endgroup$ – RustyStatistician Feb 11 '16 at 7:06
  • $\begingroup$ @RustyStatistician There would be no point in taking the expectation if there was no $\beta$. My question was if I could just plug in the mean vector of $\beta$ even though it's inside both the log and the cdf... (I don't think it's possible since those functions are not linear.) $\endgroup$ – Daeyoung Lim Feb 11 '16 at 7:09
  • $\begingroup$ I was confused because you said "I wonder if this is the same as $\log \Phi\left(x^{T}\mu\right)$" $\endgroup$ – RustyStatistician Feb 11 '16 at 7:11
  • $\begingroup$ This may help stats.stackexchange.com/questions/57715/… $\endgroup$ – RustyStatistician Feb 11 '16 at 7:19
  • 1
    $\begingroup$ Recall $g(E(X))\neq E(g(X))$ for nonlinear functions $g$, see en.wikipedia.org/wiki/Jensen%27s_inequality $\endgroup$ – Christoph Hanck Feb 11 '16 at 7:26

It should be noted that, in general, for any non-linear function $g$ of a random variable $X$, $ \mathbb{E}[g(X)] \neq g(\mathbb{E}[X]) $ . You can see this by expanding out the integrals; $$ \mathbb{E}[g(X)] = \int_X g(x)f(x)dx;\;\;\;\;g(\mathbb{E}[X])=g\bigg(\int_X xf(x)dx\bigg) $$ You could see how the two would only be equal in the case $g$ was linear.

So given the above; $$\mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right] \neq \log \Phi\left(x^{T}\mu\right)$$

You can estimate $\mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right] $ via Monte Carlo

$$ \mathbb{E}_{\beta}\left[\log \Phi \left(x^{T}\beta\right)\right]=\int_\beta \log \Phi \left(x^{T}\beta\right)f(\beta|\mu, \Sigma)d\beta \approx \frac{1}{G}\sum_{g=1}^G \log \Phi \left(x^{T}\beta^{(g)}\right) $$ where $\beta^{(1)},\beta^{(2)},...,\beta^{(G)} \sim \mathcal{N}(\mu,\Sigma)$.

If the dimension of $\beta$ is large, the above approximation may not work efficiently (it will take a very large $G$ for it to get close). Otherwise, this is a pretty standard and well accepted approach.

I am not aware of a closed form solution to the above integral. If another user does derive an analytical solution you should use that instead of the the Monte Carlo approximation above. There may also be ways to improve the above estimate with importance sampling.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.