16
$\begingroup$

Take an expectation of the form $E(f(X))$ for some univariate random variable $X$ and an entire function $f(\cdot)$ (i.e., the interval of convergence is the whole real line)

I have a moment generating function for $X$ and hence can easily calculate integer moments. Use a Taylor series around $\mu \equiv E(x)$ and then apply the expectation in terms of a series of central moments, $$ E(f(x)) = E\left(f(\mu) + f'(\mu)(x - \mu) + f''(\mu)\frac{(x - \mu)^2}{2!} +\ldots\right) $$ $$ =f(\mu) + \sum_{n=2}^{\infty} \frac{f^{(n)}(\mu)}{n!}E\left[(x - \mu)^n\right] $$ Truncate this series, $$ E_N(f(x)) = f(\mu) + \sum_{n=2}^{N} \frac{f^{(n)}(\mu)}{n!}E\left[(x - \mu)^n\right] $$

My question is: under what conditions on the random variable (and anything additional on $f(\cdot)$ as well) does the approximation of the expectation converge as I add terms (i.e. $\lim\limits_{N\to\infty}E_N(f(x)) = E(f(x))$).

Since it does not appear to converge for my case (a poisson random variable and $f(x) = x^{\alpha}$), are there any other tricks for finding approximate expectations with integer moments when these conditions fail?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ see here: stats.stackexchange.com/questions/70490/… $\endgroup$
    – Jonathan
    Commented Jul 10, 2015 at 1:34
  • $\begingroup$ @Jonathan Thank you. See my edits now that it has become clearer. Very helpful, though I couldn't quite crack it. From this, it appears that a sufficient condition for this to work is that my random variable is strongly concentrated? Though I am having trouble cracking exactly how to use Hoeffding's Inequality, etc. to compare to these notes. $\endgroup$
    – jlperla
    Commented Jul 10, 2015 at 4:10
  • $\begingroup$ What do you mean "a poisson random variable and $f(x)=x^α$"? Is that one case or two, and what is the pdf? $\endgroup$
    – Carl
    Commented Feb 22, 2018 at 4:54
  • $\begingroup$ @Carl This is a few years back, but if I remember, the variable was $x \sim Poisson(\lambda)$ for some $\lambda$ with PDF from en.wikipedia.org/wiki/Poisson_distribution. That $f(x)$ was the function I was taking the expectation over. i.e. $E(f(x))$ $\endgroup$
    – jlperla
    Commented Feb 22, 2018 at 4:58
  • $\begingroup$ Not sure what you are asking. How about that the higher moments $m_k$ of the Poisson distribution about the origin are Touchard polynomials in $\lambda$: $$m_k = \sum_{i=0}^k \lambda^i \left\{\begin{matrix} k \\ i \end{matrix}\right\},$$where the {braces} denote Stirling numbers of the second kind? $\endgroup$
    – Carl
    Commented Feb 22, 2018 at 5:16

2 Answers 2

Reset to default
4
$\begingroup$

By your assumption that $f$ is real-analytic, $$ y_n = f(\mu) + f'(\mu)(x - \mu) + f''(\mu)\frac{(x - \mu)^2}{2!} + \ldots + f^{(n)}(\mu)\frac{(x - \mu)^n}{n!} $$ converges almost surely (in fact surely) to $f(x)$.

A standard condition under which a.s. convergence implies convergence of expectation, i.e. $$ E[f(x)] = E [ \lim_{n \rightarrow \infty} y_n] = \lim_{n \rightarrow \infty} E [y_n], $$ is that $|y_n| \leq y$ a.s. for some $y$ such that $E[y] < \infty$. (Dominated Convergence Theorem.)

This condition would hold if the power series converges absolutely a.s., i.e. $$ y = \sum_{n \geq 0} |f^{(n)}(\mu)| \, \frac{|x - \mu|^n}{n!} < \infty \;\; a.s. $$ and $$ E[y] < \infty. $$

Your example of a Poisson random variable and $f(x)=x^{\alpha}$, $\alpha \notin\mathbb{Z}_+$, would suggest that the above integrability of absolute limit criterion is the weakest possible, in general.

Improve this answer
$\endgroup$
-1
$\begingroup$

The approximation will converge if the function f(x) admits to power series expansion i.e. all derivatives exist. It also will be fully achieved if derivatives of a specific threshold and above are equal to zero. You can refer to Populis[3-4] and Stark and Woods [4].

Improve this answer
$\endgroup$
2
  • $\begingroup$ "It also will be fully achieved if derivatives of a specific threshold and above are equal to zero." If the derivatives exist and are equal to zero, isn't that another way of saying polynomial? $\endgroup$
    – Acccumulation
    Commented Nov 14, 2018 at 19:12
  • 2
    $\begingroup$ This is not true. When "all derivatives exist" at the point of the power series expansion, the power series need not converge anywhere. (The standard example is the Maclaurin series of $e^{-1/x^2}.$) Another is that even when the series does converge at some point, it need not converge everywhere. A simple example is the Maclaurin series of $1/(1-x).$ When that occurs, convergence depends on the details of the random variable. For instance, suppose $X$ has any Student t distribution and consider $$1/(1-X)=1+X+X^2+\cdots+X^n+\cdots.$$ Eventually, $E(X^n)$ doesn't even exist! $\endgroup$
    – whuber ♦
    Commented Nov 14, 2018 at 19:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.