2
$\begingroup$

I am unfamiliar in statistics. My knowledge is in pure mathematics.

Suppose $n\in\mathbb{N}$, where $X$ is in the $\sigma$-algebra of Caratheodory-measurable sets such that $X\subseteq\mathbb{R}^{n}$ and $f$ is a measurable function where $f:X\to\mathbb{R}$.

Edit: For some measure $\mu$ on measurable space $(X,\Sigma)$, the average w.r.t to $\mu$, where $0<\mu(X)<\infty$, is:

$$\text{avg}_f(X)=\frac{1}{\mu(X)}\int_{X}f(x) \; d\mu$$

However, in mathematics $\mu(X)$ need not equal $1$ but I wish to know, in detail, how to get the same result using a probability measure.

Question: How does one use statistics to find the mean of $f$, using the uniform probability measure on $X$ (explained here and here in pg.32-37)?

I assume that in statistics the expected value is the same as the mean, but in textbooks, we're shown the expected value applies to the probability density function or the cumulative distribution function.

Furthermore, when the Lebesgue measure is defined $\sigma$-algebra of Lebesgue-measurable sets, which satisfy the Caratheodory criterion, I heard it's not a good idea to use the mean of $f$ w.r.t the uniform probability measure when:

  1. $X$ is countably infinite (the uniform probability measure does not exist)
  2. $X$ has infinite or zero Lebesgue Measure
  3. $f$ is an infinite number of points that covers an infinite expanse of space in general.

(Optional): How do we find the mean of $f$ for these cases?

Improve this question
$\endgroup$
7
  • $\begingroup$ @Ben Sorry, stupid mistake. $\endgroup$
    – Arbuja
    Jan 23 at 21:52
  • 1
    $\begingroup$ You apply the definition of expectation or, nearly equivalently, LOTUS. $\endgroup$
    – whuber ♦
    Jan 23 at 21:54
  • 3
    $\begingroup$ why, it's simply $\frac{1}{\mu(X)}\int f d\mu$, or am I missing something? $\endgroup$
    – John Madden
    Jan 23 at 21:55
  • 3
    $\begingroup$ When you assume a uniform distribution, you have to use a uniform distribution! If you use some other measure, you would integrate against it. $\endgroup$
    – whuber ♦
    Jan 23 at 23:50
  • 1
    $\begingroup$ RE the final three cases: Usually there is no intrinsic definition. It's tempting, for instance, to use a hypervolume to normalize a measure-zero set (such as arclength to define a uniform measure on a circle in the plane). But not all measure-zero sets have such hypervolumes and even when they do, there can be multiple conflicting ways to define a "natural" uniform measure on them. This all goes to show that probability measures are not canonically equivalent to measurable subsets. For more, read about Bertrand's Paradox. $\endgroup$
    – whuber ♦
    Jan 24 at 19:27

1 Answer 1

Reset to default
3
$\begingroup$

Note: I'm going to use the notation $\mathscr{X}$ for the set that you call $X$ and I'm going to define a random vector $\mathbf{X}$ as a uniform random vector on that set. It is better to avoid standard upper-case symbols for sets in this context and use script fonts instead. This avoids notational conflation between random variables and the support sets over which they're defined.

The expected value is fairly simple in this case. Suppose we let $\mathbf{X} \sim \text{Uniform}(\mathscr{X})$ denote a uniform random variable on the set $\mathscr{X} \subseteq \mathbb{R}^n$ and let $p_{\mathbf{X}}$ denote its density function. (If you are looking for a rigorous transition from the probability measure, the density function is a Radon-Nikodym derivative of the probability measure; the latter in this case is the Lebesgue measure over the set $\mathscr{X}$.) Then using the law of the unconscious statistician we have:

$$\begin{align} \mathbb{E}(f(\mathbf{X})) &= \int \limits_{\mathbb{R}^n} f(\mathbf{x}) \cdot p_{\mathbf{X}}(\mathbf{x}) \ d \mathbf{x} \\[6pt] &= \int \limits_{\mathbb{R}^n} f(\mathbf{x}) \cdot \frac{\mathbb{I}(\mathbf{x} \in \mathscr{X})}{|\mathscr{X}|} \ d \mathbf{x} \\[6pt] &= \frac{1}{|\mathscr{X}|} \int \limits_{\mathscr{X}} f(\mathbf{x}) \ d \mathbf{x}. \\[6pt] \end{align}$$

In practice, computation of this expected value may be complicated if the set $\mathscr{X}$ is complicated. If analytic integration does not give a closed form solution then a general and relatively simple way to compute the expected value (up to high accuracy) is with importance sampling. To do this, we produce values $\mathbf{X}_1,\mathbf{X}_2,...,\mathbf{X}_M \sim \text{IID }g$ for some density function $g$ with support $\mathscr{X} \subseteq \text{support} (g) \subseteq \mathbb{R}^n$ (hopefully with support fairly close to $\mathscr{X}$) and we use the estimator:

$$\begin{align} \hat{\mu}_{M} &\equiv \frac{\sum_{i=1}^M \mathbb{I}(\mathbf{X}_i \in \mathscr{X}) \cdot f(\mathbf{X}_i)/g(\mathbf{X}_i)}{\sum_{i=1}^M \mathbb{I}(\mathbf{X}_i \in \mathscr{X})/ g(\mathbf{X}_i)}. \end{align}$$

From the law of large numbers we can establish that $\mathbb{E}(f(\mathbf{X})) = \lim_{M \rightarrow \infty} \hat{\mu}_{M}$ so if we take $M$ to be large then we should get a reasonably good computation of the expected value of interest.

Improve this answer
$\endgroup$
11
  • $\begingroup$ Are there cases where importance sampling cannot be defined? Consider for instance fractal $\mathscr{X}$, countably infinite $\mathscr{X}$, or an infinite amount of pseudo-random, non-uniformly distributed points in space $[a,b]\times[c,d]$, where $a,b,c,d\in\mathbb{R}$ where $a\to-\infty$ and $b\to\infty$, and $c\to-\infty$ and $d\to\infty$. $\endgroup$
    – Arbuja
    Jan 24 at 3:32
  • 1
    $\begingroup$ Importance sampling essentially requires three things: (1) you need to be able to compute whether or not a point $\mathbf{x}$ is in the set $\mathscr{X}$ or not; (2) you need to be able to generate points from a density $g$ that is on a support that covers $\mathscr{X}$ but is not too much bigger than it; and (3) you have to be able to compute $f(\mathbf{x})$ and $g(\mathbf{x})$ for each point $\mathbf{x} \in \mathscr{X}$. So long as you have those things, you can implement importance sampling. $\endgroup$
    – Ben
    Jan 24 at 4:21
  • $\begingroup$ +1. It would be good to mention the problematic nature of the question when the "arbitrary set" is either not measurable or has measure zero. E.g., what if $X$ is the unit circle in the plane? $\endgroup$
    – whuber ♦
    Jan 24 at 14:53
  • $\begingroup$ @whuber I corrected my post one last time. Let me know if there’s still something wrong. $\endgroup$
    – Arbuja
    Jan 24 at 19:16
  • 1
    $\begingroup$ @Arbuja: If the CDF is differentiable then $p_\mathbf{X}$ would be its density, so there is no difference. (Also, that is not the standard mathematical notation for a derivative.) $\endgroup$
    – Ben
    Mar 6 at 21:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.