Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Let $CDF$ be the cumulative distribution function for the standard normal distribution. Let $Z$ be a standard normal random variable.

Then $CDF(Z)$ is uniformly distributed on the unit interval, so by integration we can show that $E(ln(CDF(Z))) = -1$.

My question: there an easy way to compute $E[\ln(CDF(Z + c))]$ for constant $c$?

What I'm really looking for is $E[\ln(CDF(Z + c))]$ on the $(-\infty, 0]$ interval and the $(0, \infty)$ interval.

share|improve this question
1  
Perhaps you can tell us in just a tad more detail what integration you are doing to come up with $\ln(\operatorname{cdf}(Z)) = -1$? $\operatorname{cdf}(Z)$ is a random variable taking on values in $(0,1)$, and its natural logarithm $\ln$ is also a random variable with values in $(-\infty,0)$. So, what does it mean when you say that $\ln(\operatorname{cdf}(Z)) = -1$?? –  Dilip Sarwate Feb 26 '13 at 21:14
    
Dilip, sorry for the conclusion, I meant E(ln(CDF(Z)))=−1. Or equivalently, if U is a uniformly distributed RV on the unit interval, E(ln(U)) = -1. –  garyrob Feb 26 '13 at 21:19
1  
E(ln(CDF(Z))) is equivalent to E(ln(U)) where U is uniformly distributed on (0, 1]. That can be found by integrating ln(), which is x(ln(x))-x + C, which is -1 if C is 0 x is 1. –  garyrob Feb 26 '13 at 21:31
add comment

2 Answers 2

up vote 5 down vote accepted

No,you'll have to do the integration numerically for each $c$ value.

Let $\Phi(x)$ be the Gaussian CDF function.

You want to evaluate $$ I=\int \Phi'(x-c) \ln \Phi(x) dx $$

Idea 1: this looks like a convolution; the Fourier transform of $\Phi'$ is easy enough, but that of $\ln \Phi$ is not; and even if you could take the transform of the second factor, inverting the resulting expression is unlikely to be feasible.

Idea 2: do it by parts, this works out to $$ \Phi(x-c) \ln \Phi(x) \vert^{\infty}_{\infty} - \int_{\infty}^{\infty} \frac{ \Phi(x-c)}{\Phi(x)} \Phi'(x) dx $$ the first term is conveneiently zero, but the second is no better off.

Idea 3: expand the $\ln \Phi$ term. Define $U=1-\Phi$ so that : $$ I=-\int U'(x-c) \ln [ 1-U(x)] dx $$ and then expand the logarithm as though $U$ were small. You end up with terms like $U'(x-c) U^n(x)$, which are still not easily integrable due to the shift in the argument.

At this point, it seems that the simplifications that arise in $E[ \ln \Phi(x)]$ aren't panning out, so, numerical integration is a feasible solution.

share|improve this answer
    
plugging the relevant integrand into Wolfram Alpha yielded no analytic results. –  Dave Feb 27 '13 at 0:23
    
Wolfram Alpha is only a crude test--there is plenty it cannot do. Your exposition here is much more interesting and convincing than that! (+1) –  whuber Feb 27 '13 at 14:12
    
Thank you for your thoughts. I had never tried numeric integration before, but I found scipy.integrate (docs.scipy.org/doc/scipy/reference/tutorial/integrate.html) and it did the job. And now I have another great took in my arsenal. –  garyrob Feb 27 '13 at 17:44
add comment

In short there is no "easy" way to do this, and Dave has tried a few sensible approaches. One approach perhaps worth a try is to taylor expand since your function is smooth and analytic, and we have a simple form for the moments of the standard normal:

Notice that

$L^* = \ln(\Phi(X))$

$L = \frac{\phi(x)}{\Phi(x)}$

$L' = \frac{\phi(x)^2}{\Phi(x)^2} - x \frac{\phi(x)}{\Phi(x)} = L(x)^2 - xL(x)$

$L'' = 2LL'-L-xL' = 2L^3 - 3xL^2 +(x^2-1) L$

$L''' = 6L^2L' - 6xLL'-3L^2+2xL+(x^2-1)L' = 6L^4-6xL^3-6xL^3+6x^2L^2-3L^2 + 2xL+(x^2-1)(L^2-xL)$

$ =6L^4-12xL^3+(7x^2-4)L^2 + (3x-x^3)L$

I had hoped we could spot a pattern there incolving Hermite polynomials or some-such, but I can't see one... however, because of the recurrence it is basic calculus and algebra that a computer could chunk through to arbitrary depth.

Armed with these we can now notice that we can define $X = Z + c$ so that $X \sim N(c,1)$ and then Taylor expand $L^*$ around the point c

$E[\ln(\Phi(X))] = E[\ln(\Phi(c)) + L(c) Z + L'(c)\frac{Z^2}{2} + L''(c)\frac{Z^3}{3!}+\cdots]$

Now remember that $E[Z^p] = (p-1)!!$ for even $p$ and $0$ otherwise, so we only keep every other term:

$E[\ln(\Phi(X))] = \ln(\Phi(c)) + L'(c)\frac{1}{2} + L'''(c)\frac{3!!}{4!}+ L^{(5)}(c)\frac{5!!}{6!} \cdots$

$E[\ln(\Phi(X))] = \ln(\Phi(c)) + L'(c)\frac{1}{2} + L'''(c)\frac{1}{8}+ L^{(5)}(c)\frac{1}{48} + L^{(7)}(c) \frac{1}{384} + \cdots + L^{(2k-1)}(c) \frac{1}{2^k k!} + \cdots$

(where we use that $(2k-1)!! = \frac{(2k)!}{2^k k!}$ )

So all that remains is to plug in the derivatives above and work out the total. If anyone spots a pattern then this should work exactly.


Incidentally, if you hadn't asked about $\ln(\Phi(x))$ but simply $\Phi(x)$ then we have a much simpler expression.

Carrying on from the expression above but using $\Phi$ in place of $L^*$ we have:

$E[\Phi(X)]=\Phi(c) + \phi(c) \sum_{k=1}^\infty \frac{H_{2k-1}(c)}{2^k k!}$

Where $H_n$ is the nth Hermite polynomial, and we have used the fact that $\frac{d^n \phi(x)}{dx^n}=H_n(x)\phi(x)$

Notice that inside the sum we have a polynomial over an exponential, so this is going to converge nicely.

Notice also that since $H_{2k-1}(0) = 0$ for all k then in the case where $c=0$ this colapses back to 0.5 as expected.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.