Skip to main content
55 votes

Why is the mean of the natural log of a uniform distribution (between 0 and 1) different from the natural log of 0.5?

This is another illustration of Jensen's inequality $$\mathbb E[\log X] < \log \mathbb E[X]$$ (since the function $x\mapsto \log(x)$ is strictly concave] and of the more general (anti-)property ...
Xi'an's user avatar
  • 107k
54 votes
Accepted

Why is the mean of the natural log of a uniform distribution (between 0 and 1) different from the natural log of 0.5?

Consider two values symmetrically placed around $0.5$ - like $0.4$ and $0.6$ or $0.25$ and $0.75$. Their logs are not symmetric around $\log(0.5)$. $\log(0.5-\epsilon)$ is further from $\log(0.5)$ ...
Glen_b's user avatar
  • 286k
28 votes

How to show that this integral of the normal distribution is finite?

Intuitively, the result is obvious because (a) $\phi$ is a rapidly decreasing function (its magnitude decreases at a quadratic exponential rate) and (b) $\Phi$ is bounded above and, for negative $x,$ ...
whuber's user avatar
  • 329k
20 votes
Accepted

Monte Carlo simulations for arbitrary functions

Draw $n$ pairs $(x,y)$, iid uniformly distributed in the unit square. Count how many of these pairs satisfy $y<x^2$, let this number be $k$. Then $$\mathbb P(Y<X^2) = \int_{[0,1]^2} \mathbb I_{y&...
Stephan Kolassa's user avatar
16 votes

KL divergence between which distributions could be infinity

What happens to $D_{KL}(p \parallel q)$ when $p(x)$ and/or $q(x)$ is zero? In a strict sense, the log of zero is undefined because there's no value of $x$ such that $e^x = 0$. But, the definition of ...
user20160's user avatar
  • 32.9k
14 votes
Accepted

Finding the slope at different points in a sigmoid curve

Your question is very broad. There are many ways to do this, even without assuming a specific function. For the following I assume that you have a good reason to use the Gompertz model. First let's ...
Roland's user avatar
  • 6,721
14 votes
Accepted

When Can Integration and Expectation be Exchanged?

Generally speaking, the expected value of an integral is an iterated integral, and so the normal mathematical rules for interchange of integrals apply. To see this more clearly, we first note that ...
Ben's user avatar
  • 129k
14 votes
Accepted

Limit of Integration of continuous function

Clearly, the integral can be rewritten as $E[f(Y_n)]$, where $Y_n = \frac{1}{n}(X_1 + \cdots + X_n)$, and $X_1, \ldots, X_n \text{ i.i.d.} \sim U(0, 1)$. By (weak) law of large numbers, we have $Y_n \...
Zhanxiong's user avatar
  • 20.8k
14 votes
Accepted

Is it wrong to say that a Riemann sum is an unbiased estimate of an integral?

It seems that you are swapping two different concepts here. The concepts are unbiased and consistent, which are properties of an estimator. A sequence of estimators $(T_n)_{n=1}^\infty$ is said to be ...
Lucas Prates's user avatar
  • 1,223
14 votes

Expected loss function from bias variance trade off (integral help)

This is a functional derivative. Employing the notation in my explanation at https://stats.stackexchange.com/a/236159/919, let $\mathcal L$ be the functional, let $h$ be any (integrable) function, and ...
whuber's user avatar
  • 329k
13 votes

Monte Carlo simulations for arbitrary functions

The representation of$$\mathfrak I = \int_0^1 x^2\,\text dx$$as an expectation of a random variable is quite open, in that the choice of a Uniform (0,1) variable $U$ such that$$\mathfrak I = \mathbb E[...
Xi'an's user avatar
  • 107k
12 votes
Accepted

Gaussian Marginal Likelihood

We want to solve $$ \int \mathcal N(y | Wx, \beta^{-1} I) \mathcal N(x | 0, I) dx $$ $$ = \frac{\beta^{D/2}}{(2\pi)^{D/2}} \cdot \frac{1}{(2\pi)^{q/2}}\int \exp \left(-\frac \beta 2 || y - Wx ||^2 - \...
jld's user avatar
  • 20.6k
12 votes

Why is the mean of the natural log of a uniform distribution (between 0 and 1) different from the natural log of 0.5?

It is worthwhile to note that if $X \sim \operatorname{Uniform}(0,1)$, then $-\log X \sim \operatorname{Exponential}(\lambda = 1)$, so that $\operatorname{E}[\log X] = -1$. Explicitly, $$f_X(x) = \...
heropup's user avatar
  • 5,456
12 votes

Expected loss function from bias variance trade off (integral help)

Starting from the objective $$\min_{\hat f}\mathbb E_D\left[\iint (y-\hat f(x))^2 p(x,y)\,\text dx\text dy \right]$$ the derivation of the optimal function$$\hat f: x\longmapsto \hat f(x)$$follows ...
Xi'an's user avatar
  • 107k
11 votes
Accepted

Compute median of continuous distribution using integrate() in R

CAUTION! As was pointed out and explained by whuber in the comments, the current code below does not check if it is fed a density that integrates to one (or to some other finite value which we could ...
Christoph Hanck's user avatar
11 votes
Accepted

How to deduce $ \mathbb{E}(\sqrt{X}) < \infty \implies\int_{\mathbb{R}^+} (1 - F(x))^2 dx < \infty,~X$ being a non-negative integrable rv?

Interesting problem. The proof goes as follows: let $X, X_1, X_2$ i.i.d. $\sim F$, then \begin{align} \int_0^\infty (1 - F(x))^2dx = E[\min(X_1, X_2)] \leq E[\sqrt{X_1}\sqrt{X_2}] = (E[\sqrt{X}])^2 ...
Zhanxiong's user avatar
  • 20.8k
10 votes
Accepted

Variance of the integral of a stochastic process

Interchanging the order of integration and expectation you get $$E(I)=E\int_0^L X(t) dt = \int_0^L EX(t) dt = \int_0^L \mu dt = L\mu $$ and similarly, the second moment of $I$ becomes \begin{align} E(...
Jarle Tufto's user avatar
  • 11.3k
9 votes
Accepted

Expressing a marginal probability using copulas

The issue of notation seems crucial. I propose, therefore, to disambiguate the ubiquitous and overloaded "$f$" by means of subscripts. Thus, $f_{XYZ}$ will be the full density function and ...
whuber's user avatar
  • 329k
9 votes

Plain English explanation of Ito's integral?

I think you're confusing the Lebesgue integral with Itô calculus. They are related concepts. I'll explain. Lebesgue vs Riemann The simplest explanation of the difference between Lebesgue and Riemann ...
Aksakal's user avatar
  • 61.9k
9 votes
Accepted

Plain English explanation of Ito's integral?

Ito's integral has nothing to do with the area under a curve and no connection with rectangles, random or otherwise. Let me try to share a motivation in the simplest language I can manage. There's ...
Flounderer's user avatar
  • 10.8k
9 votes

Monte Carlo simulations for arbitrary functions

My purpose here is to show a Riemann approximation for $\int_0^1 x^2\, dx = 1/3$ with enough rectangles to approximate the integral. Then to do a Monte Carlo integration in which uniformly chosen ...
BruceET's user avatar
  • 57k
8 votes
Accepted

Integral of a CDF

For cdfs $F$ of distributions with supports on $(0,a)$, $a$ being possibly $+\infty$, a useful representation of the expectation is $$\mathbb{E}_F[X]=\int_0^a x \text{d}F(x)=\int_0^a \{1-F(x)\}\text{...
Xi'an's user avatar
  • 107k
8 votes
Accepted

Constant of Laplace approximation

This example is indeed rather poorly conducted and full of typos, apologies from one author!!! First, there is no genuine $n$ (or related sample size) in the picture so the Laplace approximation ...
Xi'an's user avatar
  • 107k
8 votes
Accepted

Weighted Average and Expectation in machine learning

When$$\int p(x)\text dx=1$$, $$\mathbb E_p[f(X)]=\dfrac{\int f(x)p(x)\,\text dx}{\int p(x)\,\text dx}$$ The notion is rarely used in probability books, as it does not help (and further depends on the ...
Xi'an's user avatar
  • 107k
8 votes

How to show that this integral of the normal distribution is finite?

Here is a self-contained elementary argument, by a comparison with the Laplace distribution. We show $$\int_{-\infty}^{\infty}\frac{\phi(x)^2}{\Phi(x)}dx<\frac{1}{2\sqrt{\pi}}+\sqrt{\frac\pi2}\...
Matt F.'s user avatar
  • 5,342
8 votes

Recurrence formula for the moments of a half-gaussian distribution (on R+)

Let $z = (x-\mu)/\sigma$ and write $$I_m = \int_0^\infty x^m\, \exp\left(-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2\right)\,\mathrm d x = \int_{-\mu/\sigma}^\infty \left(\sigma z + \mu\right)^m e^...
whuber's user avatar
  • 329k
7 votes

$\int_a^b \mathbb{P}(\mathrm{d}x) = \int_a^b \mathbb{P}(x)\mathrm{d}x$?

In general, the $\mathbb{P}$-integral of a measurable function $X$ on $\left(\Omega, \mathcal{A}, \mathbb{P} \right)$ is given by $$\int_{\Omega} X(\omega) \ d \mathbb{P}(\omega)$$ which is also ...
JohnK's user avatar
  • 20.8k
7 votes

Deriving the moments of Student-t distributions

There are various ways to find the moments of the T-distribution, but the simplest method is to use the mixture representation using the normal distribution. If $T$ has a Student's T distribution ...
Ben's user avatar
  • 129k
7 votes
Accepted

Importance Sampling

Pretty close. Remember that exponentials have support on the positive reals. $$ \frac{1}{N}\sum_i f(x_i)/g(x_i) \to E_g[f(x)/g(x)] = \int_0^{\infty}f(x)/g(x) g(x)dx = \int_0^{\infty}f(x) dx \neq \...
Taylor's user avatar
  • 21.2k
7 votes

Why does integrating a probability density function give probability?

The probability density function is actually defined by this requirement. In introductory probability courses, you will see the density function defined as the (Riemann) derivative of the cumulative ...
Ben's user avatar
  • 129k

Only top scored, non community-wiki answers of a minimum length are eligible