30 votes
Accepted

Proof that variance is always greater than or equal to zero

Go to your definition of variance: $$ \operatorname{Var}(X) = \int(x-\mu)^2f(x)\,dx $$ The $(x-\mu)^2$ component is non-negative, and the $f(x)$ component is non-negative, so the integrand, $(x-\mu)^...
Dave's user avatar
  • 62.4k
12 votes

Oracle Inequality : In basic terms

I'll try to explain it in linear case. Consider the linear model $$Y_i=\sum_{j=1}^{p} \beta_jX_{i}^{(j)}+\epsilon_i, i=1,...,n. $$ When $p \leq n$ (number of independent variables less or equal then ...
D.G's user avatar
  • 116
12 votes

Taylor expansion in Hoeffding's Lemma proof

This is the mean-value form of Taylor's theorem: $$f(x)=f(0)+xf'(0)+\frac{x^2}{2}f''(c)$$ where $c$ is between $0$ and $x$ Take $x=h$ and $c=h\theta$
Thomas Lumley's user avatar
9 votes
Accepted

How can I establish an inequality between $|\frac1n \sum_{i=1}^nX_i|$ and $\frac1n\sum^n_{i=1}|X_i|$ where $X_i \sim N(0,1)$?

Assuming independent $X_i$, the mean $\frac{1}{n}\sum X_i$ is also normal, i.e. $N(0,1/n)$. Absolute value of it is Half-normal, which has mean $E[Y_1]=\frac{\sigma\sqrt{2}}{\sqrt{\pi}}=\sqrt{\frac{2}{...
gunes's user avatar
  • 57.3k
9 votes
Accepted

Prove that Kurtosis is at least one more than the square of the skewness

This demonstration uses the clever (yet elementary) method employed in Fisher's reference Burnside & Panton vol. II, section 142. It assumes a familiarity with the basics of determinants: their ...
whuber's user avatar
  • 323k
8 votes

Proof that variance is always greater than or equal to zero

As for your question regarding complex numbers, the variance is defined as being the expectation of the absolute value, or modulus, squared of the deviation from the mean. If the absolute value is not ...
Acccumulation's user avatar
8 votes

What is the correlation between a random variable and its probability integral transform?

If we assume $\mathbb E^F[X]=0$ then \begin{align} \mathbb E^F[XF(X)]&= \frac{1}{2}\int F(x)\{1-F(x)\}\,\text dx \end{align} Indeed, assuming the pdf $f$ is associated with the cdf $F$, \...
Xi'an's user avatar
  • 106k
8 votes
Accepted

Property of two independent Beta distribution

It does not seem to be a correct conjecture. It seems your condition is that the mode for $X$ is greater than the mode for $Y$. Since in non-symmetric Beta distributions, the mode is not equal to the ...
Henry's user avatar
  • 39.6k
8 votes
Accepted

Sign of Correlation between $X$ and $f(X)$ for strictly monotonic $f$

Let $f$ be strictly increasing. Then $$\operatorname{Cov}(X, f(X)) =\mathbb E[Xf(X) ]-\mathbb E[X]\mathbb E[f(X) ]=\mathbb E[(X-\mathbb E[X])(f(X) -f(\mathbb E[X]))].\tag 1\label 1$$ Now $$X\gtreqless\...
User1865345's user avatar
  • 8,292
7 votes
Accepted

What is the correlation between a random variable and its probability integral transform?

When $X$ has a uniform distribution on the interval $[-\sqrt{3},\sqrt{3}]$ it has unit variance and its distribution function on this interval is $$F_X(x) = \frac{1}{2\sqrt{3}}(\sqrt{3}+x),$$ whence ...
whuber's user avatar
  • 323k
7 votes
Accepted

Expectation of first of moment of symmetric r.v. in terms of variance

You can actually develop your first example a little more to refute this conjecture, that is put some mass at $0$. Let $X$ be the random variable such that $P[X = 0] = 1 - p$, $P[X = -1] = P[X = 1] = ...
Zhanxiong's user avatar
  • 18.8k
6 votes
Accepted

Zero mean unit variance random variables bound on probability

I will focus on the case where $|\rho| < 1$. We are given that $\mathbb{E}[X_1] = \mathbb{E}[X_2]=0$, $\operatorname{Var}[X_1]=\operatorname{Var}[X_2]=1, $ and the correlation is $\rho$. From ...
Siong Thye Goh's user avatar
6 votes
Accepted

Lower Bound on $E[\frac{1}{X}]$ for positive symmetric distribution

Let's try the usual preliminaries: simplify by choosing appropriate units of measurement and exploiting the symmetry assumption. Reframing the question Change the units of $X$ so that its mean is $m=1$...
whuber's user avatar
  • 323k
6 votes
Accepted

Prove that $E(X\ln X)\le E X E\ln X$

Remark: the sign seems to be flipped. Here is how we can prove if $X> 0$, then $$E(X \ln X) \ge E(X) E(\ln X)$$ We can apply Jensen's inequality twice. $x\ln x$ is convex, hence we have,$$E[...
Siong Thye Goh's user avatar
6 votes

How can I establish an inequality between $|\frac1n \sum_{i=1}^nX_i|$ and $\frac1n\sum^n_{i=1}|X_i|$ where $X_i \sim N(0,1)$?

Answer: Whatever the distribution of $X_1,...,X_n$, $$\mathbb{E} Y_2 \geq \mathbb{E} Y_1.$$ Details: For any $n$ numbers $X_1,..., X_n$ it is true that $$ \sum_i |X_i| \geq |\sum_i X_i|$$ and ...
Konstantin's user avatar
  • 1,331
6 votes
Accepted

Upper bound for absolute third central moment

Let $p\ge 1.$ The $L^p$ norm of a random variable is defined as $$|X|_p = \left(E[|X|^p]\right)^{1/p}.$$ Minkowski's Inequality says this norm satisfies the triangle inequality. Apply it to the ...
whuber's user avatar
  • 323k
6 votes
Accepted

Is it true that $\langle X^4\rangle \ge 3 \langle X^2\rangle^2$?

Both of the inequalities you assert are false For a random variable with zero mean, the moment quantity $\langle X^4 \rangle /\langle X^2 \rangle^2$ is the kurtosis of the distribution, which has a ...
Ben's user avatar
  • 125k
5 votes
Accepted

Variance and covariance inequality

One way to do it is to proceed from the answer you linked in comments, at the second last line (at this point no properties of the normal have been used). Note that $Y$ is just a standardized $X$; i.e....
Glen_b's user avatar
  • 283k
5 votes

Prove that Kurtosis is at least one more than the square of the skewness

Consider the quadratic form \begin{align} Q(a, ~b,~c) & := \frac1n\sum \left(a + xb+ x^2 c\right)^2 \\ &= a^2\cdot v_0 + b^2\cdot v_2 + c^2\cdot v_4+ 2ab\cdot v_1+ 2bc\cdot v_3+ 2ca\cdot v_2,\...
User1865345's user avatar
  • 8,292
5 votes
Accepted

Testing a Null Hypothesis Nested by the Alternative Hypothesis

By definition (see e.g. Jun Shao "Mathematical Statistics", 2nd edition, chapter 6, first paragraph) of statistical hypothesis tests, the null and the alternative hypothesis must be disjoint....
frank's user avatar
  • 10.8k
5 votes

Is it true that $\langle X^4\rangle \ge 3 \langle X^2\rangle^2$?

An easy counterexample is a two point distribution $P(X = \pm 1) = 1/2$, for which $E[X] = 0$, $E[X^2] = E[X^4] = 1$. Hence $1 = E[X^4] < 3(E[X^2])^2 = 3$. This example also showed the related ...
Zhanxiong's user avatar
  • 18.8k
4 votes
Accepted

How does Chebyshev's inequality imply $P(X ≥ k) ≤ 1/(σk)^2$?

Your second form of the inequality is wrong. The initial form of Chebychev's inequality (for a random variable $X$ with zero mean) is: $$\mathbb{P}(|X| \geqslant k \sigma) \leqslant \frac{1}{k^2}.$$ ...
Ben's user avatar
  • 125k
4 votes

If $X < a$, $EX < a$?

\begin{align} EX &= \int x\cdot \mathbf{1}_{x< a} + x\cdot\mathbf{1}_{x\geq a} \ dP \\ &= \int x\cdot \mathbf{1}_{x< a}\ dP\\ &< \int a\cdot \mathbf{1}_{x< a}\ dP \\ &= a, \...
Christian Chapman's user avatar
4 votes
Accepted

Variance of the reciprocal of a strictly positive random variable

Here is an answer (to a related question) which provides a valid generalization of $\mathbb{E}\left[\frac{1}{X}\right] \geq \frac{1}{\mathbb{E}[X]}$. What is the relation between $\mathbb{E}X^r$ and ...
Mark L. Stone's user avatar
4 votes

Variance of the reciprocal of a strictly positive random variable

First, @Dilip Sarwate comments, for such ratio variables mean and variance often do not exist and then there is little to expect. For a detailed discussion of this see I've heard that ratios or ...
kjetil b halvorsen's user avatar
4 votes
Accepted

Variance inequality

This is just Jensen's inequality for the square root function $ f(x) = \sqrt{x} $. Since $ f $ is concave, you know $ \mathbb E[f(X)] \leq f(\mathbb E[X]) $. Applying this to the random variable $ (X -...
Ege Erdil's user avatar
  • 211
4 votes
Accepted

Use Chebyshev's inequality to find a lower bound of a Chi-Square Distribution

Chebyshev's inequality is an inequality. It tells you that $P(|X-40| ≤ 20)\geq 0.8$. So if the probability that $20\leq X\leq 60$ is at least $0.8$, the inequality is satisfied. It is: ...
Glen_b's user avatar
  • 283k
4 votes
Accepted

Chebyshev's inequality for Pareto distribution (3 sigma rule)

Let $X$ be a random variable with mean $\mu$ and standard deviation $\sigma$. By simple probability rules, we have $$P\left(\mu-3\sigma \leq X \leq \mu+3\sigma\right) = P(X \leq \mu + 3\sigma) - P(X \...
knrumsey's user avatar
  • 7,772
4 votes
Accepted

Hoeffding type concentration result for the inverse of a sum of iid random variables

It is possible to come up with something like this. Here is an example. The first thing to note is that, $$\hat{\mu}^{-1}=n/S_n$$ Where $S_n = X_1 + \dots + X_n$. Now under the assumption that the $...
Ariel's user avatar
  • 2,477
4 votes

How to calculate lower bound on $P \left[|Y| > \frac{|\lambda|}{2} \right]$?

Edit : This answer applies to the original question, that was : "for ANY random variable Y, what is the lower bound of (formula)" The lower bound is 0. Let's take $(Y_{n})$ a series ...
Adrien's user avatar
  • 159

Only top scored, non community-wiki answers of a minimum length are eligible