53

This is another illustration of Jensen's inequality $$\mathbb E[\log X] < \log \mathbb E[X]$$ (since the function $x\mapsto \log(x)$ is strictly concave] and of the more general (anti-)property that the expectation of the transform is not the transform of the expectation when the transform is not linear (plus a few exotic cases). (Most of my undergraduate ...


52

Consider two values symmetrically placed around $0.5$ - like $0.4$ and $0.6$ or $0.25$ and $0.75$. Their logs are not symmetric around $\log(0.5)$. $\log(0.5-\epsilon)$ is further from $\log(0.5)$ than $\log(0.5+\epsilon)$ is. So when you average them you get something less than $\log(0.5)$. Similarly, if you take a teeny interval around a collection of ...


42

Here are some general hints on solving this question: You have a sequence of continuous IID random variables which means they are exchangeable. What does this imply about the probability of getting a particular order for the first $n$ values? Based on this, what is the probability of getting an increasing order for the first $n$ values? It is possible to ...


37

Here's a famous example: Let $X$ take value $2^k$ with probability $2^{-k}$, for each integer $k\ge1$. Then $X$ takes values in (a subset of) the positive integers; the total mass is $\sum_{k=1}^\infty 2^{-k}=1$, but its expectation is $$E(X) = \sum_{k=1}^\infty 2^k P(X=2^k) = \sum_{k=1}^\infty 1 = \infty. $$ This random variable $X$ arises in the St. ...


34

You are right to be skeptical of this approach. The Taylor series method does not work in general, although the heuristic contains a kernel of truth. To summarize the technical discussion below, Strong concentration implies that the Taylor series method works for nice functions Things can and will go dramatically wrong for heavy-tailed distributions or not-...


34

For the more restricted question Why is a biased standard deviation formula typically used? the simple answer Because the associated variance estimator is unbiased. There is no real mathematical/statistical justification. may be accurate in many cases. However, this is not necessarily always the case. There are at least two important aspects of ...


31

The letters that derive from $\mu$ include the Roman M and the Cyrillic М. Hence considering that the word "mean" starts with an $m$ the choice seems relatively straightforward given an already existing tradition to use greek letters in mathematical abbrevation. To satisfy certain individuals craving for actual historical research and assuming that the ...


31

Take two positive iid Cauchy variates $Y_1,Y_2$ with common density $$f(x)=\frac{2}{\pi}\frac{\mathbb I_{x>0}}{1+x^2}$$ and infinite expectation. The minimum variate $\min(Y_1,Y_2)$ then has density $$g(x)=\frac{8}{\pi^2}\frac{\pi/2-\arctan(x)}{1+x^2}\mathbb I_{x>0}$$ Since (by L'Hospital's rule) $$\frac{\pi/2-\arctan(x)}{1+x^2} \equiv \frac{1}{x^3}$$ ...


29

The result extends to the $k$th moment of $X$ as well. Here is a graphical representation:


29

can it be 1/E(X)? No, in general it can't; Jensen's inequality tells us that if $X$ is a random variable and $\varphi$ is a convex function, then $\varphi(\text{E}[X]) \leq \text{E}\left[\varphi(X)\right]$. If $X$ is strictly positive, then $1/X$ is convex, so $\text{E}[1/X]\geq 1/\text{E}[X]$, and for a strictly convex function, equality only occurs if $X$ ...


28

I would like to offer a very simple, intuitive explanation. It amounts to looking at a picture: the rest of this post explains the picture and draws conclusions from it. Here is what it comes down to: when there is a "probability mass" concentrated near $X=0$, there will be too much probability near $1/X\approx \pm \infty$, causing its expectation ...


27

Imagine that you are in Paris in 1654 and you and your friend are observing a gambling game based on sequential rolling of a six sided dice. Now, gambling is highly illegal and busts by the gendarme are quite frequent, and to be caught at a table with stacks of livre is to almost surely guarantee a lengthy stint in the Chateau d'If. To get around this you ...


26

Recall that $e^x\geq 1+x$ $E\left[e^{Y}\right]=e^{ E(Y)} E\left[e^{Y- E(Y)}\right]\geq e^{E(Y)} E\left[1+{Y- E(Y)}\right] = e^{E(Y)}$ So $e^{E(Y)}\leq E\left[e^{Y}\right] $ Now letting $Y=\ln X$, we have: $e^{E(\ln X)}\leq E\left[e^{\ln X}\right]=E(X)$ now take logs of both sides $E[\ln (X)]\leq\ln[E(X)]$ Alternatively: $\ln X = \ln X - \ln \mu+\ln\...


25

$$T=7/3+(8+T)/3+(12+T)/3=9+2T/3$$ $$T/3=9$$ $$T=27$$ From start point, each of 3 paths are equally possible. Two paths lead you back to start point.


25

Call the next chests as $X_1,X_2$. With $0.4$ probability, our new variable is $X_1+X_2$ and with $0.6$ probability, it is $1$. So, $$\begin{align}E[X^2]&=0.4\times E[(X_1+X_2)^2]+0.6\times1^2\\&=0.4\times E[X_1^2+X_2^2+2X_1X_2]+0.6\\&=0.4\times(2E[X^2]+2E[X]^2)+0.6\\&=0.8\times E[X^2]+7.8\rightarrow E[X^2]=39\rightarrow\operatorname{var}(X)=...


24

Here is one I found at https://www.qualitydigest.com/inside/quality-insider-article/problems-skewness-and-kurtosis-part-one.html# which I find nice and reproduced in R: an inverse Burr or Dagum distribution with shape parameters $k=0.0629$ and $c=18.1484$: $$g(x) = ckx^{-(c+1)}[1+x^{-c}]^{-(k+1)}$$ It has mean 0.5387, standard deviation 0.2907, skewness 0....


24

Because the bootstrapped statistic is one further abstraction away from your population parameter. You have your population parameter, your sample statistic, and only on the third layer you have the bootstrap. The bootstrapped mean value is not a better estimator for your population parameter. It's merely an estimate of an estimate. As $n \rightarrow \infty$...


24

It may be helpful to think of rectangles. Imagine you have the chance to get land for free. The size of the land will be determined by (a) one realization of the random variable or (b) two realizations of the same random variable. In the first case (a), the area will be a square with the side length being equal to the sampled value. In the second case (b), ...


24

In short: No. There are several properties of a probability distribution that need not affect its mean and variance, but do determine its shape. Skew & Kurtosis For example, a Poisson distribution with $\lambda = 1$ has expected value $\lambda = 1$ and variance $\lambda = 1$. So does a normal distribution with $\mu = 1$ and $\sigma^2 = 1$. An example ...


24

Let's find a general solution for independent variables $X$ and $Y$ having CDFs $F_X$ and $F_Y,$ respectively. This will give us useful clues into what's going on, without the distraction of computing specific integrals. Let $Z=\min(X,Y).$ Then, from basic axioms and definitions, we can work out that for any number $z,$ $$\eqalign{ F_Z(z) &= \Pr(Z\le ...


23

The trick is that $\mathbb{E}(\hat{\theta}) - \theta$ is a constant.


22

Since $$\begin{align} \operatorname{Cov}(Y, Z) &= E[(Y - E[Y])(Z - E[Z])] \\ &= E[(Y - {\textstyle \int}_0^{2\pi} \sin x \;dx)(Z - {\textstyle \int}_0^{2\pi} \cos x \;dx)] \\ &= E[(Y - 0)(Z - 0)] \\ &= E[YZ] \\ &= \int_0^{2\pi} \sin x \cos x \;dx \\ &= 0 , \end{align}$$ the correlation must also be 0.


20

A normally distributed variable $X$ with mean $\mu$ and variance $\sigma^2$ has the same distribution as $\sigma Z + \mu$ where $Z$ is a standard normal variable. All you need to know about $Z$ is that its cumulative distribution function is called $\Phi$, it has a probability density function $\phi(z) = \Phi^\prime(z)$, and that $\phi^\prime(z) = -z \phi(...


20

There is a general rule to use Greek letters for parameters and Latin letters for statistics. Why $\mu$? Well, the word 'mean' in English starts with M and $\mu$ sounds like M. But also, per Google translate: Latin: Media French: Moyenne Spanish: Media German: Mittel Dutch: Midden


20

Actually, it's relatively simple to obtain formulas for the entire distribution as well as an easy procedure to compute any moment of it. For $n=1,2,3,\ldots,$ let $f_n(p) = \Pr(X=n)$ with $p=0.6.$ Define $$F_p(t) = f_1(p)t + f_2(p)t^2 + \cdots + f_n(p)t^n + \cdots$$ (the probability generating function). The problem asserts $$F_p(t) = p\,t + (1-p)F_p^2(...


19

This can be answered using the geometric distribution as follows: The number of failures k - 1 before the first success (heads) with a probability of success p ("heads") is given by: $$p(X=k)=(1−p)^{k−1}p$$ with k being the total number of tosses including the first 'heads' that terminates the experiment. And the expected value of X for a given p is $1/p=...


19

For a Normal distribution, the expected value, a.k.a. the mean, equals the mode. In general, not only is the expected value not only not the most likely (or at highest density), but it may have no chance of occurring. For instance, consider the random variable X which equals 0 or 2, each with probability 0.5. Then EX = 1, but the expected value,1, has 0 ...


19

Let $K$ be some random variable. In your problem, $K$ is number of times you flip before getting heads. Let $f(k)$ be some payoff function. In your problem $f(k) = 2^k$. Let $f(K)$ be the payoff You're saying that a reasonable valuation of the gamble $f(K)$ is given by $f(\mathrm{E[}K])$. This is an entirely ad-hoc, rather unprincipled heuristic. ...


19

This one (maybe surprisingly) can be done with easy elementary operations (employing Richard Feynman's favorite trick of differentiating under the integral sign with respect to a parameter). We are supposing $X$ has a $\Gamma(\alpha,\beta)$ distribution and we wish to find the expectation of $Y=\log(X).$ First, because $\beta$ is a scale parameter, its ...


19

You can think about your problem as a Markov chain, i.e., a set of states with certain transition probabilities between states. You start in one state (all cards face up) and end up in an absorbing state (all cards face down). Your question is about the expected number of steps until you reach that absorbing state, either for a single chain, or for the ...


Only top voted, non community-wiki answers of a minimum length are eligible