22
$\begingroup$

I know $E(aX+b) = aE(X)+b$ with $a,b $ constants, so given $E(X)$, it's easy to solve. I also know that you can't apply that when its a nonlinear function, like in this case $E(1/X) \neq 1/E(X)$, and in order to solve that, I've got to do an approximation with Taylor's. So my question is how do I solve $E(\ln(1+X))$?? do I also approximate with Taylor?

$\endgroup$
  • 4
    $\begingroup$ Yes you can apply the delta method in this case. $\endgroup$ – Michael Chernick Sep 29 '12 at 23:45
  • 4
    $\begingroup$ You should also look into the Jensen Inequality. $\endgroup$ – kjetil b halvorsen Sep 30 '12 at 19:58
26
$\begingroup$

In the paper

Y. W. Teh, D. Newman and M. Welling (2006), A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation, NIPS 2006, 1353–1360.

a second order Taylor expansion around $x_0=\mathbb{E}[x]$ is used to approximate $\mathbb{E}[\log(x)]$:

$$ \mathbb{E}[\log(x)]\approx\log(\mathbb{E}[x])-\frac{\mathbb{V}[x]}{2\mathbb{E}[x]^2} \>. $$

This approximation seems to work pretty well for their application.

Modifying this slightly to fit the question at hand yields, by linearity of expectation,

$$ \mathbb{E}[\log(1+x)]\approx\log(1+\mathbb{E}[x])-\frac{\mathbb{V}[x]}{2(1+\mathbb{E}[x])^2} \>. $$

However, it can happen that either the left-hand side or the right-hand side does not exist while the other does, and so some care should be taken when employing this approximation.

$\endgroup$
  • 3
    $\begingroup$ Interestingly, This can be used to get an approximation to the digamma function. $\endgroup$ – probabilityislogic Oct 3 '12 at 22:53
5
$\begingroup$

Suppose that $X$ has probability density $f_X$. Before you start approximating, remember that, for any measurable function $g$, you can prove that $$ E[g(X)]=\int g(X)\,dP = \int_{-\infty}^\infty g(x)\,f_X(x)\,dx \, , $$ in the sense that if the first integral exists, so does the second, and they have the same value.

$\endgroup$
  • 1
    $\begingroup$ If the second integral exists. It needs not to. Take Cauchy distribution and $g(x)=x^2$. $\endgroup$ – mpiktas Sep 30 '12 at 13:34
  • $\begingroup$ I would add a second layer of pedantry by saying that you actually need $E[|g(X)|]<\infty$ for the expectation to be well defined. $\endgroup$ – probabilityislogic Oct 3 '12 at 22:45
  • 2
    $\begingroup$ @mpiktas - This expectation actually does exist but it is infinite. A better example is $g(x)=x$ for the Cauchy distribution. This expectation depends on how the lower and upper limits of integration tend to infinity. $\endgroup$ – probabilityislogic Oct 3 '12 at 22:49
  • 2
    $\begingroup$ @prob: No, you don't need that condition in your first comment, and even in a situation that may be very relevant to this question! (+1 to your second comment, though, which was something I had been meaning to comment on as well.) $\endgroup$ – cardinal Oct 3 '12 at 22:51
  • 2
    $\begingroup$ @prob: It is sufficient, but if you compare your first comment to your second one, you'll see why it's not necessary! :-) $\endgroup$ – cardinal Oct 3 '12 at 23:02
5
$\begingroup$

Also, if you don't need an exact expression for $\text{E}[\log(X + 1)]$, oftentimes the bound given by Jensen's inequality is good enough: $$ \log [\text{E}(X) + 1] \geq\text{E}[\log(X + 1)] $$

$\endgroup$
  • $\begingroup$ just wanted to add: if no direct calculation is possible and you look at a single variable $X$, jensen's inequality is about your only option to get any useful result. while the suggested taylor approximation may indeed work in praxis, there is no theoretical justification which could be used to motivate the deletion of the remainder terms. (that being said: keep in mind that the infinitely taylor series of ln(1+x) converges only in a radius |x|<1) anyway.) $\endgroup$ – chRrr Nov 23 '17 at 17:18
  • $\begingroup$ I think it shoud be $\ge$ since $\log$ is concave down. $\endgroup$ – Deep North Jun 14 '18 at 13:05
4
$\begingroup$

There are two usual approaches:

  1. If you know the distribution of $X$, you may be able to find the distribution of $\ln(1+X)$ and from there find its expectation; alternatively you may be able to use the law of the unconscious statistician directly (that is, integrate $\ln(1+x) f_{X}(x)$ over the domain of $x$).

  2. As you suggest, if you know the first few moments you can compute a Taylor approximation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.