34
$\begingroup$

I know $E(aX+b) = aE(X)+b$ with $a,b $ constants, so given $E(X)$, it's easy to solve. I also know that you can't apply that when its a nonlinear function, like in this case $E(1/X) \neq 1/E(X)$, and in order to solve that, I've got to do an approximation with Taylor's. So my question is how do I solve $E(\ln(1+X))$?? do I also approximate with Taylor?

$\endgroup$
2
  • 5
    $\begingroup$ Yes you can apply the delta method in this case. $\endgroup$ Sep 29, 2012 at 23:45
  • 6
    $\begingroup$ You should also look into the Jensen Inequality. $\endgroup$ Sep 30, 2012 at 19:58

4 Answers 4

41
$\begingroup$

In the paper

Y. W. Teh, D. Newman and M. Welling (2006), A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation, NIPS 2006, 1353–1360.

a second order Taylor expansion around $x_0=\mathbb{E}[x]$ is used to approximate $\mathbb{E}[\log(x)]$:

$$ \mathbb{E}[\log(x)]\approx\log(\mathbb{E}[x])-\frac{\mathbb{V}[x]}{2\mathbb{E}[x]^2} \>. $$

This approximation seems to work pretty well for their application.

Modifying this slightly to fit the question at hand yields, by linearity of expectation,

$$ \mathbb{E}[\log(1+x)]\approx\log(1+\mathbb{E}[x])-\frac{\mathbb{V}[x]}{2(1+\mathbb{E}[x])^2} \>. $$

However, it can happen that either the left-hand side or the right-hand side does not exist while the other does, and so some care should be taken when employing this approximation.

$\endgroup$
1
  • 3
    $\begingroup$ Interestingly, This can be used to get an approximation to the digamma function. $\endgroup$ Oct 3, 2012 at 22:53
10
$\begingroup$

Also, if you don't need an exact expression for $\text{E}[\log(X + 1)]$, oftentimes the bound given by Jensen's inequality is good enough: $$ \log [\text{E}(X) + 1] \geq\text{E}[\log(X + 1)] $$

$\endgroup$
3
  • $\begingroup$ just wanted to add: if no direct calculation is possible and you look at a single variable $X$, jensen's inequality is about your only option to get any useful result. while the suggested taylor approximation may indeed work in praxis, there is no theoretical justification which could be used to motivate the deletion of the remainder terms. (that being said: keep in mind that the infinitely taylor series of ln(1+x) converges only in a radius |x|<1) anyway.) $\endgroup$
    – chRrr
    Nov 23, 2017 at 17:18
  • 1
    $\begingroup$ I think it shoud be $\ge$ since $\log$ is concave down. $\endgroup$
    – Deep North
    Jun 14, 2018 at 13:05
  • $\begingroup$ @chRrr the taylor expansion with remainder theorem is a pretty good theoretical justification to motivate ignoring the remainder terms I think. $\endgroup$ Mar 8, 2021 at 23:11
8
$\begingroup$

There are two usual approaches:

  1. If you know the distribution of $X$, you may be able to find the distribution of $\ln(1+X)$ and from there find its expectation; alternatively you may be able to use the law of the unconscious statistician directly (that is, integrate $\ln(1+x) f_{X}(x)$ over the support of $x$).

  2. As you suggest, if you know the first few moments you can compute a Taylor approximation. However, as whuber suggests in comments, see here

$\endgroup$
2
  • $\begingroup$ Re Taylor approximation: see my comment stats.stackexchange.com/questions/38296/… about the problems of lack of convergence. $\endgroup$
    – whuber
    Jan 13 at 20:51
  • $\begingroup$ Yeah, important thing to note in this circumstance $\endgroup$
    – Glen_b
    Jan 14 at 2:24
7
$\begingroup$

Suppose that $X$ has probability density $f_X$. Before you start approximating, remember that, for any measurable function $g$, you can prove that $$ E[g(X)]=\int g(X)\,dP = \int_{-\infty}^\infty g(x)\,f_X(x)\,dx \, , $$ in the sense that if the first integral exists, so does the second, and they have the same value.

$\endgroup$
6
  • 1
    $\begingroup$ If the second integral exists. It needs not to. Take Cauchy distribution and $g(x)=x^2$. $\endgroup$
    – mpiktas
    Sep 30, 2012 at 13:34
  • $\begingroup$ I would add a second layer of pedantry by saying that you actually need $E[|g(X)|]<\infty$ for the expectation to be well defined. $\endgroup$ Oct 3, 2012 at 22:45
  • 2
    $\begingroup$ @mpiktas - This expectation actually does exist but it is infinite. A better example is $g(x)=x$ for the Cauchy distribution. This expectation depends on how the lower and upper limits of integration tend to infinity. $\endgroup$ Oct 3, 2012 at 22:49
  • 2
    $\begingroup$ @prob: No, you don't need that condition in your first comment, and even in a situation that may be very relevant to this question! (+1 to your second comment, though, which was something I had been meaning to comment on as well.) $\endgroup$
    – cardinal
    Oct 3, 2012 at 22:51
  • 2
    $\begingroup$ @prob: It is sufficient, but if you compare your first comment to your second one, you'll see why it's not necessary! :-) $\endgroup$
    – cardinal
    Oct 3, 2012 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.