# Tag Info

54

Informally, a probability distribution defines the relative frequency of outcomes of a random variable - the expected value can be thought of as a weighted average of those outcomes (weighted by the relative frequency). Similarly, the expected value can be thought of as the arithmetic mean of a set of numbers generated in exact proportion to their ...

53

This is another illustration of Jensen's inequality $$\mathbb E[\log X] < \log \mathbb E[X]$$ (since the function $x\mapsto \log(x)$ is strictly concave] and of the more general (anti-)property that the expectation of the transform is not the transform of the expectation when the transform is not linear (plus a few exotic cases). (Most of my undergraduate ...

52

Consider two values symmetrically placed around $0.5$ - like $0.4$ and $0.6$ or $0.25$ and $0.75$. Their logs are not symmetric around $\log(0.5)$. $\log(0.5-\epsilon)$ is further from $\log(0.5)$ than $\log(0.5+\epsilon)$ is. So when you average them you get something less than $\log(0.5)$. Similarly, if you take a teeny interval around a collection of ...

40

Here are some general hints on solving this question: You have a sequence of continuous IID random variables which means they are exchangeable. What does this imply about the probability of getting a particular order for the first $n$ values? Based on this, what is the probability of getting an increasing order for the first $n$ values? It is possible to ...

38

Here's a famous example: Let $X$ take value $2^k$ with probability $2^{-k}$, for each integer $k\ge1$. Then $X$ takes values in (a subset of) the positive integers; the total mass is $\sum_{k=1}^\infty 2^{-k}=1$, but its expectation is $$E(X) = \sum_{k=1}^\infty 2^k P(X=2^k) = \sum_{k=1}^\infty 1 = \infty.$$ This random variable $X$ arises in the St. ...

34

For the more restricted question Why is a biased standard deviation formula typically used? the simple answer Because the associated variance estimator is unbiased. There is no real mathematical/statistical justification. may be accurate in many cases. However, this is not necessarily always the case. There are at least two important aspects of ...

32

You are right to be skeptical of this approach. The Taylor series method does not work in general, although the heuristic contains a kernel of truth. To summarize the technical discussion below, Strong concentration implies that the Taylor series method works for nice functions Things can and will go dramatically wrong for heavy-tailed distributions or not-...

30

The letters that derive from $\mu$ include the Roman M and the Cyrillic М. Hence considering that the word "mean" starts with an $m$ the choice seems relatively straightforward given an already existing tradition to use greek letters in mathematical abbrevation. To satisfy certain individuals craving for actual historical research and assuming that the ...

28

Consider discrete distributions. One that is supported on $k$ values $x_1, x_2,\ldots, x_k$ is determined by non-negative probabilities $p_1, p_2,\ldots, p_k$ subject to the conditions that (a) they sum to 1 and (b) the skewness coefficient equals 0 (which is equivalent to the third central moment being zero). That leaves $k-2$ degrees of freedom (in the ...

28

can it be 1/E(X)? No, in general it can't; Jensen's inequality tells us that if $X$ is a random variable and $\varphi$ is a convex function, then $\varphi(\text{E}[X]) \leq \text{E}\left[\varphi(X)\right]$. If $X$ is strictly positive, then $1/X$ is convex, so $\text{E}[1/X]\geq 1/\text{E}[X]$, and for a strictly convex function, equality only occurs if $X$ ...

26

Imagine that you are in Paris in 1654 and you and your friend are observing a gambling game based on sequential rolling of a six sided dice. Now, gambling is highly illegal and busts by the gendarme are quite frequent, and to be caught at a table with stacks of livre is to almost surely guarantee a lengthy stint in the Chateau d'If. To get around this you ...

26

Recall that $e^x\geq 1+x$ $E\left[e^{Y}\right]=e^{ E(Y)} E\left[e^{Y- E(Y)}\right]\geq e^{E(Y)} E\left[1+{Y- E(Y)}\right] = e^{E(Y)}$ So $e^{E(Y)}\leq E\left[e^{Y}\right]$ Now letting $Y=\ln X$, we have: $e^{E(\ln X)}\leq E\left[e^{\ln X}\right]=E(X)$ now take logs of both sides $E[\ln (X)]\leq\ln[E(X)]$ Alternatively: $\ln X = \ln X - \ln \mu+\ln\... 25 First I think that you may be confusing "statistics" meaning a collection of numbers or other facts describing a group or situation, and "statistics" meaning the science of using data and information to understand the world in the face of variation (others may be able to improve on my definitions). Statisticians use both senses of the word, so it is not ... 25 I would like to offer a very simple, intuitive explanation. It amounts to looking at a picture: the rest of this post explains the picture and draws conclusions from it. Here is what it comes down to: when there is a "probability mass" concentrated near$X=0$, there will be too much probability near$1/X\approx \pm \infty$, causing its expectation to be ... 25 $$T=7/3+(8+T)/3+(12+T)/3=9+2T/3$$ $$T/3=9$$ $$T=27$$ From start point, each of 3 paths are equally possible. Two paths lead you back to start point. 25 Call the next chests as$X_1,X_2$. With$0.4$probability, our new variable is$X_1+X_2$and with$0.6$probability, it is$1. So, \begin{align}E[X^2]&=0.4\times E[(X_1+X_2)^2]+0.6\times1^2\\&=0.4\times E[X_1^2+X_2^2+2X_1X_2]+0.6\\&=0.4\times(2E[X^2]+2E[X]^2)+0.6\\&=0.8\times E[X^2]+7.8\rightarrow E[X^2]=39\rightarrow\operatorname{var}(X)=... 24 Here is one I found at https://www.qualitydigest.com/inside/quality-insider-article/problems-skewness-and-kurtosis-part-one.html# which I find nice and reproduced in R: an inverse Burr or Dagum distribution with shape parameters k=0.0629 and c=18.1484:g(x) = ckx^{-(c+1)}[1+x^{-c}]^{-(k+1)}$$It has mean 0.5387, standard deviation 0.2907, skewness 0.... 24 Because the bootstrapped statistic is one further abstraction away from your population parameter. You have your population parameter, your sample statistic, and only on the third layer you have the bootstrap. The bootstrapped mean value is not a better estimator for your population parameter. It's merely an estimate of an estimate. As n \rightarrow \infty... 24 It may be helpful to think of rectangles. Imagine you have the chance to get land for free. The size of the land will be determined by (a) one realization of the random variable or (b) two realizations of the same random variable. In the first case (a), the area will be a square with the side length being equal to the sampled value. In the second case (b), ... 24 In short: No. There are several properties of a probability distribution that need not affect its mean and variance, but do determine its shape. Skew & Kurtosis For example, a Poisson distribution with \lambda = 1 has expected value \lambda = 1 and variance \lambda = 1. So does a normal distribution with \mu = 1 and \sigma^2 = 1. An example ... 23 Since$$\begin{align} \operatorname{Cov}(Y, Z) &= E[(Y - E[Y])(Z - E[Z])] \\ &= E[(Y - {\textstyle \int}_0^{2\pi} \sin x \;dx)(Z - {\textstyle \int}_0^{2\pi} \cos x \;dx)] \\ &= E[(Y - 0)(Z - 0)] \\ &= E[YZ] \\ &= \int_0^{2\pi} \sin x \cos x \;dx \\ &= 0 , \end{align}$$the correlation must also be 0. 22 The trick is that \mathbb{E}(\hat{\theta}) - \theta is a constant. 20 Actually, it's relatively simple to obtain formulas for the entire distribution as well as an easy procedure to compute any moment of it. For n=1,2,3,\ldots, let f_n(p) = \Pr(X=n) with p=0.6. Define$$F_p(t) = f_1(p)t + f_2(p)t^2 + \cdots + f_n(p)t^n + \cdots$$(the probability generating function). The problem asserts$$F_p(t) = p\,t + (1-p)F_p^2(... 20 Take two positive iid Cauchy variates with density $$f(x)=\frac{2}{\pi}\frac{\mathbb I_{x>0}}{1+x^2}$$ and infinite expectation. The minimum variate\min(Y_1,Y_2)$then has density $$g(x)=\frac{8}{\pi^2}\frac{\pi/2-\arctan(x)}{1+x^2}\mathbb I_{x>0}$$ Since (by L'Hospital's rule) $$\frac{\pi/2-\arctan(x)}{1+x^2} \equiv \frac{1}{x^3}$$ at infinity, the ... 19 Let$K$be some random variable. In your problem,$K$is number of times you flip before getting heads. Let$f(k)$be some payoff function. In your problem$f(k) = 2^k$. Let$f(K)$be the payoff You're saying that a reasonable valuation of the gamble$f(K)$is given by$f(\mathrm{E[}K])$. This is an entirely ad-hoc, rather unprincipled heuristic. ... 19 There is a general rule to use Greek letters for parameters and Latin letters for statistics. Why$\mu$? Well, the word 'mean' in English starts with M and$\mu$sounds like M. But also, per Google translate: Latin: Media French: Moyenne Spanish: Media German: Mittel Dutch: Midden 18 This can be answered using the geometric distribution as follows: The number of failures k - 1 before the first success (heads) with a probability of success p ("heads") is given by: $$p(X=k)=(1−p)^{k−1}p$$ with k being the total number of tosses including the first 'heads' that terminates the experiment. And the expected value of X for a given p is$1/p=...

18

A normally distributed variable $X$ with mean $\mu$ and variance $\sigma^2$ has the same distribution as $\sigma Z + \mu$ where $Z$ is a standard normal variable. All you need to know about $Z$ is that its cumulative distribution function is called $\Phi$, it has a probability density function $\phi(z) = \Phi^\prime(z)$, and that $\phi^\prime(z) = -z \phi(... 18 For a Normal distribution, the expected value, a.k.a. the mean, equals the mode. In general, not only is the expected value not only not the most likely (or at highest density), but it may have no chance of occurring. For instance, consider the random variable X which equals 0 or 2, each with probability 0.5. Then EX = 1, but the expected value,1, has 0 ... 18 Let's construct all possible examples of random variables$X$for which$E[X]E[1/X]=1\$. Then, among them, we may follow some heuristics to obtain the simplest possible example. These heuristics consist of giving the simplest possible values to all expressions that drop out of a preliminary analysis. This turns out to be the textbook example. Preliminary ...

Only top voted, non community-wiki answers of a minimum length are eligible