Questions tagged [inequality]

Use this tag if you question involves the use of an inequality. The inequality may have probabilistic origins or be a purely mathematical inequality. Do not use for measures of inequality, for instance income inequality. For that use the [diversity] tag.

Filter by
Sorted by
Tagged with
17 votes
1 answer
4k views

Oracle Inequality : In basic terms

I'm going through a paper that uses oracle inequality to prove something but I'm unable to understand what it is even trying to do. When I searched online about 'Oracle Inequality', some sources ...
Wolcott's user avatar
  • 171
11 votes
2 answers
15k views

Bounds on Cov(X, Y) given Var(X), Var(Y)?

I'm generating random multivariate normal data using the rmultnorm() function in R, which allows users to specify a vector of $k$ population means and a $k \times k$...
RobertF's user avatar
  • 6,104
10 votes
2 answers
4k views

Proof that variance is always greater than or equal to zero

It is common knowledge that: $$\begin{equation}\label{3} Var(X) \geq 0 \end{equation}$$ for every random variable $X$. Despite this, I do not remember seeing a formal proof of this. Is there a proof ...
OliverVD's user avatar
  • 139
9 votes
2 answers
643 views

Prove that Kurtosis is at least one more than the square of the skewness

Wikipedia claims it, and on reading the paper that it linked I found that the proof that was written there was quite difficult. Is there a simple proof possible for this identity? The proof given in ...
Martund's user avatar
  • 535
8 votes
2 answers
530 views

What is the correlation between a random variable and its probability integral transform?

Are there known bounds on the $\operatorname{cor}(X,F(X))$? $X$ is a random variable with CDF $F(X)$. Let $X$ have a fixed variance, for example $\operatorname{var}(X)=1$. What $X$ can maximize or ...
sayda's user avatar
  • 309
8 votes
1 answer
3k views

KL divergence bounds square of L1 norm

In Cover & Thomas, Elements of Information Theory, at the section on Conditional Limit Theorem (11.6), it is proved that the KL divergence bounds the $\cal{L}_1$-norm from above, $\frac{1}{2\ln2}\...
Shlomi A's user avatar
  • 195
7 votes
1 answer
200 views

Property of two independent Beta distribution

I have been working with beta-bernoulli posteriors recently. Is it true that if $X,Y$ are independent rvs with $X \sim Beta(a_1+1,b_1+1)$ and $Y \sim Beta(a_2+1,b_2+1)$ then $\mathbb{P}(X>Y)>0.5$...
Sushant Vijayan's user avatar
7 votes
1 answer
251 views

Lower Bound on $E[\frac{1}{X}]$ for positive symmetric distribution

Let $X$ be positive random variable and its distribution is symmetric about its mean value $m$. Then $$ E\left[\frac{1}{X}\right] \geq \frac{1}{m} + \frac{\sigma^2}{m^3}, $$ where $\sigma^2$ is ...
Ethan's user avatar
  • 463
7 votes
0 answers
141 views

Inequalities on Fisher Information / expected second derivative?

Under some regularity conditions we can compute fisher information as $ - \mathbb{E}_{\theta_0} [\frac{\partial}{\partial \theta^2} \ln f(x;\theta_0)] $ I was wondering if there are some kind of ...
a_student's user avatar
  • 291
6 votes
2 answers
217 views

How to calculate lower bound on $P \left[|Y| > \frac{|\lambda|}{2} \right]$?

Let $Y$ be a random variable such that $E[Y] = \lambda$, $\lambda \in \mathbb{R}$ and $E[Y^2]<\infty$. The problem is to find a lower bound on the probability $$ P \left[|Y| > \frac{|\lambda|}{...
Bhisham's user avatar
  • 319
6 votes
3 answers
137 views

Is $P(|X_1|>k)\le P(|X_2|> k)$ when $X_i\sim N(\mu_i,\sigma^2)$ and $|\mu_2| \ge |\mu_1|$?

Suppose $X_1\sim N(\mu_1,\sigma^2)$ and $X_2\sim N(\mu_2,\sigma^2)$ where $\mu_2\ge \mu_1$. Since $\mu_2\ge \mu_1$, based on a characterization of stochastic ordering, we can say that $$P(X_1>c)\le ...
StubbornAtom's user avatar
  • 11.2k
6 votes
1 answer
163 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
alfalfa's user avatar
  • 631
5 votes
2 answers
157 views

How can I establish an inequality between $|\frac1n \sum_{i=1}^nX_i|$ and $\frac1n\sum^n_{i=1}|X_i|$ where $X_i \sim N(0,1)$?

Let $X_1, \ldots , X_n$ be a random sample from a $N(0,1)$ population. Define $Y_1=|\frac1n \sum_{i=1}^nX_i|$ and $Y_2=\frac1n\sum^n_{i=1}|X_i|$. Find a relationship between $E(Y_1)$ and $E(Y_2)$. I ...
Ron Snow's user avatar
  • 2,129
5 votes
1 answer
513 views

Taylor expansion in Hoeffding's Lemma proof

Hoeffding's Lemma proof uses Taylor expansion with this statement: From Taylor's theorem, for some $ 0\leq \theta \leq 1$ $ L(h) = L(0) + h L'(0) + \frac{1}{2} h^2 L''(h\theta) \leq \frac{1}{8}h^2 $ ...
Tavakoli's user avatar
  • 151
5 votes
1 answer
679 views

Upper bound for absolute third central moment

Suppose $X\in \mathbb{R}$ is a random variable with expected value $\mathbb{E}X = \mu$. I ran across a proof which uses the inequality $$ \mathbb{E}[|X - \mu|^3] \leq 2^3 \mathbb{E}|X|^3. $$ Can ...
safelyanonymous's user avatar
5 votes
1 answer
444 views

Hoeffding type concentration result for the inverse of a sum of iid random variables

Consider a collection of $n$ i.i.d. Bernoulli random variables $\{ X_i \}_{i=1}^{n}$ with $\mathbb{E}[X_i] = \mu$. Then, if $\hat{\mu}$ is the mean of the $n$ random variables, i.e. if, $$\hat{\mu} = \...
ijuneja's user avatar
  • 185
5 votes
2 answers
3k views

Inequality on variance of sum

I want to prove that $$\operatorname{Var}\left(\sum\limits_{i=1}^m{X_i}\right) \leq m\sum\limits_{i=1}^m{\operatorname{Var}(X_i)} \,. \>$$ A too complicated proof is to write $$ a_{ij}=\sqrt {Cov(...
Ronald's user avatar
  • 69
5 votes
1 answer
1k views

Difference of two KL-divergence

The Kullback-Leibler (KL) divergence between two distributions $P$ and $Q$ is defined as $$\mbox{KL}(P \| Q) = \mathbb{E}_P\left[\ln \frac{\mbox{d}P}{\mbox{d}Q}\right].$$ My question is that suppose ...
Matics's user avatar
  • 301
4 votes
1 answer
1k views

Zero mean unit variance random variables bound on probability

Let $X_1, X_2$ be zero-mean, unit variance Random Variables with correlation coefficient $\rho$ then $$ P(|X_1|\le\epsilon,|X_2|\le\epsilon) \ge 1-\epsilon^{-2}(1+\sqrt{1-\rho^2}) $$ I tried to used ...
Ethan's user avatar
  • 463
4 votes
2 answers
256 views

A question involving directional derivatives and differential inequalities

This is a follow-up question to A question about copulas and directional derivatives. Since no answer was given, I am going to precise the definition of copula. I am interested in proving (or ...
user avatar
4 votes
1 answer
264 views

Variance inequality

Why does the following hold? For a random variable X with finite necessary moments, $E(|X|) \leq \sqrt{Var(X)}+|E(X)|$
Henry's user avatar
  • 115
4 votes
2 answers
6k views

Kullback-Leibler divergence lower bound

Are there any (nontrivial) lower bounds on the KL-divergence between two densities? Informally, I am trying to study problems where $f$ is some target density, and I want to show that if $g$ is chosen ...
JohnA's user avatar
  • 712
4 votes
1 answer
2k views

Expressing conditional probability with inequality in condition

Is there a convenient way to express $$ p(A\leq a |B\leq b \land C\leq c), $$ when all I've got is an expression for $$ p(A\leq a |B = b \land C = c), $$ $$ p(A\leq a |B = b), $$ and $$ ...
Tommy L's user avatar
  • 1,553
4 votes
1 answer
1k views

Does the local triangle inequality holds for Kullback-Leibler divergence

Does the local triangle inequality holds for the Kullback-Leibler divergence? For the local triangle inequality, I mean the $$ d(\theta', \theta) + d(\theta'', \theta) \geq A d(\theta', \theta'') $$ ...
Steve's user avatar
  • 287
4 votes
1 answer
86 views

Sum of values with different probabilities

Suppose I have the following linear expression: $S = x_1 + x_2+ \dots + x_n$, in which each $x_i$ can only assume the following values: -2, -1, 0, 1, 2 whose probabilities are 0.1, 0.2, 0.2, 0.25, 0....
donut's user avatar
  • 263
4 votes
0 answers
274 views

Proving an inequality for CDF's

I am working on a proof to show that given $x_1, x_2,\ldots,x_k$ random variables with a joint pdf and joint CDF, show that $$ 1-\sum_{i=1}^k \overline{F_i(x_i)} \leq F(x_1,x_2,\ldots,x_k) \leq \min_i ...
chow's user avatar
  • 93
4 votes
0 answers
326 views

Which concentration inequalities apply when moments are infinite?

I have 2 questions: Suppose I have a finite mean but an infinite variance for a discrete distribution w/support $\{1,2,\dots\}$. Is there any probability inequality tighter than Markov in this case? ...
Lucas Roberts's user avatar
3 votes
2 answers
155 views

Is it true that $\langle X^4\rangle \ge 3 \langle X^2\rangle^2$?

Consider a real random variable $X$ with zero mean. Does the following inequality hold in general? $$\langle X^4\rangle \ge 3 \langle X^2\rangle^2$$ I'm not sure how to prove this or if a counter-...
a06e's user avatar
  • 4,420
3 votes
1 answer
124 views

Sign of Correlation between $X$ and $f(X)$ for strictly monotonic $f$

This question is a follow up to this question. Suppose $f$ is strictly increasing. Can we say $$\text{Cov}(X,f(X))\geq 0?$$ Ben's answer on the aforementioned linked post can be extended to show the ...
Golden_Ratio's user avatar
3 votes
1 answer
195 views

Expectation of first of moment of symmetric r.v. in terms of variance

Let $X$ be a symmetric random variable with bounded moments and standard deviation $\sigma$. I want to lower-bound $\mathbb E[|X|]$ in terms of $\sigma$. Here is the formal conjecture; I wonder if ...
AvidLearner's user avatar
3 votes
1 answer
317 views

Testing a Null Hypothesis Nested by the Alternative Hypothesis

Consider a parameter of interest $\beta \in \mathcal{B}$, and two hypotheses $$H_0:\;\beta\in\mathcal{B_0}\quad versus \quad H_1:\;\beta\in\mathcal{B}_1$$ where $\mathcal{B}_0\cup\mathcal{B_1}=\...
M.C. Park's user avatar
  • 925
3 votes
1 answer
86 views

Proving an inequality

I am working with this problem. Can someone take a look at it? "Suppose $X$ and $Y$ are arbitrary random variables with finite second moments. you are told $P(X+Y=0)<1$. Then is $$ \sqrt{E[(X+Y)^...
Joel Sinofsky's user avatar
3 votes
1 answer
592 views

Use Chebyshev's inequality to find a lower bound of a Chi-Square Distribution

I'm trying to solve the following exercise but I'm not sure if what I'm doing is right. "Let $X$ be an r.v. distributed as $\chi_{40}^{2}$. Use Tchebichev’s inequality in order to find a lower ...
Stephen Dedalus's user avatar
3 votes
1 answer
217 views

If $X < a$, $EX < a$?

If a r.v. $X < a$, does it imply $EX < a$? If not, why is it different from what I know: If a r.v. $X \leq a$, it implies $EX \leq a$, proved by replacing $X$ with $a$ as the integrand. Note ...
Tim's user avatar
  • 19.4k
3 votes
1 answer
2k views

Proof of the multivariate Cramer-Rao inequality

I search a detailed proof of the multivariate Cramer-Rao inequality in the general case where the estimator is not necessarily unbiased. Let $T(X)$ be an estimator of the parameter $\theta\in\mathbb{R}...
Michael Baudin's user avatar
3 votes
1 answer
1k views

When is Jensen's Inequality strict?

For a homework problem, I have to prove that for a random sample $X_1, \ldots, X_n$, drawn from a population with finite variance $\sigma^2$, with sample mean $\bar{x}$ and sample variance $s^2$, that ...
wzbillings's user avatar
3 votes
1 answer
185 views

Need help to understand Feller's statement "whenever $r$th moment exists so do all preceding moments"

I am reading the book of Feller called "An Introduction to Probability Theory and Its Applications, Vol I" (third edition, page 227) and am stuck at the moment he explains the notion of variance of a ...
mtmrv's user avatar
  • 33
3 votes
1 answer
144 views

Expectation of the Inverse of the Sample Mean vs. Inverse of Expectation of the Sample Mean

For an iid sample of $n$ realizations of random variable $X$ with $\mathbb{E}\left[X\right] = \mu > 0$, let $\bar{X}_n$ denote its sample mean. Is there a distribution for $X$ such that $\mathbb{E}\...
Jakob J's user avatar
  • 43
3 votes
2 answers
99 views

Boundary of $E\left[\frac{\prod_{i=1}^n x_i}{\prod_{i=1}^n x_i+\prod_{i=n+1}^m x_i}\right]$

Suppose $X_i$ are i.i.d. In addition, $X_i>0$ and $E[X_i]>1$. Suppose $E[X_i]$ is known, could we find upper bound or lower bound for the following expectation: $$ E\left[\frac{\prod_{i=1}^n x_i}...
Claucisco's user avatar
3 votes
1 answer
303 views

L2 SVM (squared hinge) theory

The linear L2 SVM can be intuitively understood as \begin{equation} \text{minimize } f(\boldsymbol{w}) = \frac{1}{2} \Vert\boldsymbol{w}\Vert^2_2 + C \sum_{i=1}^m \xi_i^2 \tag{1} \end{equation} ...
Tim Mak's user avatar
  • 1,182
3 votes
1 answer
60 views

Implications of zero limiting variance

Assume that I have a sequence of random variables $X_1, X_2, \dots$ with means $\mu_1, \mu_2, \dots$ such that $\lim_{n \to \infty} \operatorname{Var}(X_n) = 0$. Can I claim that for large enough $n$ ...
Oleksandr Shchur's user avatar
3 votes
1 answer
47 views

Estimator based on inequality data

$X_i \sim N(\mu, \sigma^2)$ (iid), $i = 1,2,...,N$, I want to estimate $\theta = (\mu, \sigma^2)$. Problem is, I don't observe $x_i$. For each $i$, I only observe $(a_i, b_i)$, and I know that $a_i &...
Jessica's user avatar
  • 1,231
3 votes
0 answers
94 views

Question about the Probability of Error for Joint-Typicality Tests

Given a set of codewords $\boldsymbol{x}_i$ with $i=1,\cdots,2^{nR}$ where $R$ is the rate of the code. The codewords are transmitted over a Gaussian channel $Y = X + W$ with $X\sim\mathcal{N}(0,A^2)$ ...
silverlining's user avatar
2 votes
2 answers
882 views

Variance of the reciprocal of a strictly positive random variable

In this post it is stated that due to Jensen's inequality the expected value of the reciprocal of a strictly postive random variable $X$ will satisfy: $$\mathbb{E}\left[\frac{1}{X}\right] \geq \frac{...
egg's user avatar
  • 1,205
2 votes
3 answers
2k views

Cauchy Schwarz inequality proof using discriminant

I know the proof but I'm unclear on one thing. Cauchy-Schwarz inequality: Given X,Y are random variables, the following holds: $$ (E[XY])^2 \le E[X^2]E[Y^2] $$ Proof Let $$ u(t) = E[(tX - Y)^2] $$ ...
s5s's user avatar
  • 685
2 votes
1 answer
58 views

Distributive property of probabilistic inequalities involving random variables on both sides

Can I break down $P(h \geq (A + B)$, given all $ A,B,h$ are all random variables. Will the following rule works? $$P[h \geq (A + B)] = P(h\geq A) + P(h\geq B)$$ Actually, in one of my mathematical ...
SJa's user avatar
  • 534
2 votes
1 answer
894 views

Need mathematical steps for Hoeffding's Inequality applied to Bernoulli Distribution

I am trying to understand Hoeffiding's Inequality in Machine Learning and I am referring to WikiPedia for it. Hoeffding's Inequality is defined as follows: $ P(|\hat{\theta} - \theta)| \ge \epsilon) \...
xabzakabecd's user avatar
  • 3,455
2 votes
1 answer
57 views

Moment inequality: $E\mid X_1 X_2 X_3\mid \leq (E(\mid X_1\mid^3)+E(\mid X_2\mid^3)+E(\mid X_3\mid^3))/3 $ for zero-mean r.v.'s?

Let $X_1, X_2, X_3$ be zero mean random variables and assume $E(\mid X_i \mid ^{4+\delta})\leq C, i=1,2,3$ where $C$ is a constant and $\delta>0$ some positive small constant. How can I show that ...
Celine Harumi's user avatar
2 votes
1 answer
766 views

Unusual Markov inequality for normal distribution

I'm trying to answer the following question from Larry Wassermans book on statistical inference. My question is how did they arrive at the Markov bound, it does not seem like the normal form of the ...
Iltl's user avatar
  • 467
2 votes
2 answers
2k views

Expectation Inequality with indicator function

When I read proof of Chebyshev's inequality, I came across the problem. At first, the proof is : \begin{align*} P(|X-r|\geq k\sigma) &= E(\chi_{|X-\mu| \geq k\sigma}) \\ &= E(\chi_{\left( \...
maso's user avatar
  • 1,359