The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [inequality]

Use this tag if you question involves the use of an inequality. The inequality may have probabilistic origins or be a purely mathematical inequality. Do not use for measures of inequality, for instance income inequality. For that use [tag:diversity].

Filter by
Sorted by
Tagged with
3
votes
2answers
114 views
+100

A question involving directional derivatives and differential inequalities

This is a follow-up question to A question about copulas and directional derivatives. Since no answer was given, I am going to precise the definition of copula. I am interested in proving (or ...
1
vote
0answers
17 views

Extreme Value Theory - Determining the positive normalising constant in the Extremal Types Theorem

I am working through the following question and cannot seem to work out how the final result is obtained from the last inequality involving $a_n$. Can someone shed some light?
0
votes
0answers
26 views

How do I solve a linear inequality system ($X\beta+b<0$)?

Given a low-dimension linear regression problem $\mathbf{y}=\mathbf{X}\beta + \epsilon$, we can easily estimate $\beta$ with $(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^Ty$. However, the problem seems ...
1
vote
0answers
56 views

Some inequality problem, which is larger?

There is a set of n random numbers whose sum equals 1, i.e., $$w_a=\{w_{a1},\; w_{a2},\; \ldots,\; w_{an}\}$$ where $$ w_{a1} + w_{a2}+ \ldots+ w_{an} =1$$ and $$0 < \{ w_{a1}, w_{a2}, \ldots, w_{...
3
votes
1answer
59 views

Need help to understand Feller's statement “whenever $r$th moment exists so do all preceding moments”

I am reading the book of Feller called "An Introduction to Probability Theory and Its Applications, Vol I" (third edition, page 227) and am stuck at the moment he explains the notion of variance of a ...
0
votes
0answers
20 views

Conditioning and linear MSE

Let $\sigma_{X|Y}^2$ denote the linear mean squared error in estimating $X$ from $Y$. Then is it always true that additional conditioning cannot increase the LLSE? In other words, is this true? $$ \...
1
vote
1answer
37 views

Moment inequality: $E\mid X_1 X_2 X_3\mid \leq (E(\mid X_1\mid^3)+E(\mid X_2\mid^3)+E(\mid X_3\mid^3))/3 $ for zero-mean r.v.'s?

Let $X_1, X_2, X_3$ be zero mean random variables and assume $E(\mid X_i \mid ^{4+\delta})\leq C, i=1,2,3$ where $C$ is a constant and $\delta>0$ some positive small constant. How can I show that ...
1
vote
1answer
32 views

Unusual Markov inequality for normal distribution

I'm trying to answer the following question from Larry Wassermans book on statistical inference. My question is how did they arrive at the Markov bound, it does not seem like the normal form of the ...
0
votes
1answer
68 views

Variance and covariance inequality

Given a real-valued random variable $X$, is $$2\mathbb E[X] \mathrm{Var}(X) \geq \mathrm{Cov}(X, X^2)$$ true? Any pointers for how to tackle this problem would be immensely helpful.
5
votes
1answer
70 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
2answers
251 views

Proving efficiency of OLS over GLS

I'm trying to prove the efficiency of OLS over GLS when the covariance matrix of the error $\varepsilon$ is mistakenly assumed to be $\sigma^2\Sigma$ instead of $\sigma^2 I$. After deriving the ...
3
votes
0answers
43 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
0answers
34 views

Can I test for inequality in H0 using chi square test?

Let's say I want to test whether an $n$-sided dice is not too unfair. In the standard chi-square test we test the zero-hypothesis $$ H_0\colon (p_1,\dots p_n) = (1/n,\dots,1/n) ,\quad\text{i.e.,}\quad ...
2
votes
1answer
33 views

Estimator based on inequality data

$X_i \sim N(\mu, \sigma^2)$ (iid), $i = 1,2,...,N$, I want to estimate $\theta = (\mu, \sigma^2)$. Problem is, I don't observe $x_i$. For each $i$, I only observe $(a_i, b_i)$, and I know that $a_i &...
2
votes
2answers
39 views

Does an inequality hold as an expectation over a probability distribution?

Suppose I have to functions $f(x)$ and $g(x)$ such that $$ f(x) \leq g(x) \quad \forall x. $$ For a distribution $\pi(x)$ on $x$, is it necessarily true that $$ E_\pi[f(x)] \leq E_\pi[g(x)]? $$ My ...
0
votes
1answer
129 views

A different proof for KL divergence non-negativity

KL divergence's non-negativity can be proved in many ways. One could use the inequality $\log x \leq x - 1$ as a main step in the proof, another one could leverage the property of concave of the ...
2
votes
3answers
133 views

Cauchy Schwarz inequality proof using discriminant

I know the proof but I'm unclear on one thing. Cauchy-Schwarz inequality: Given X,Y are random variables, the following holds: $$ (E[XY])^2 \le E[X^2]E[Y^2] $$ Proof Let $$ u(t) = E[(tX - Y)^2] $$ ...
4
votes
0answers
74 views

Proving an inequality for CDF's

I am working on a proof to show that given $x_1, x_2,\ldots,x_k$ random variables with a joint pdf and joint CDF, show that $$ 1-\sum_{i=1}^k \overline{F_i(x_i)} \leq F(x_1,x_2,\ldots,x_k) \leq \min_i ...
0
votes
2answers
50 views

Summation of squared x_i if summation of x_i is 1

How to prove "If $\sum_{i=1}^n x_i=1$, then $\sum_{i=1}^n x_i^2>1/n$"? I'm thinking about $Var(x_i)=E(x_i^2)-[E(x_i)]^2=\frac{1}{n}\sum_{i=1}^n x_i^2-1/n^2\ge0$. Is that correct?
1
vote
1answer
28 views

Help understanding a probability inequality

I'm working throught Wasserman's "All of Statistics" book. When proving convergence of random variables/distributions in chapter 5, he lists the following inequality: $$F_n(x) = \mathbb{P}(X_n\le x)=\...
0
votes
1answer
147 views

Prove that $E(X\ln X)\le E X E\ln X$ [closed]

I want to prove it using Jensen inequality, so I need to prove that $g(x)=x\ln x$ is a convex function, which means $$g\left(\frac{a+b}{2}\right)\le \frac{1}{2}\left(g(a)+g(b)\right).$$ How can I ...
1
vote
1answer
36 views

How does Chebyshev's inequality imply $P(X ≥ k) ≤ 1/(σk)^2$?

I am aware that Chebyshev's inequality $P(X ≥ kσ) ≤ 1/k^2$ can also be written as $P(X ≥ k) ≤ 1/(σk)^2$, but I do not understand the math to convert between these forms. Could someone explain/point me ...
2
votes
1answer
154 views

An inequality giving a sharper bound than that given by the Chebyshev's?

Let $X > 0$ be a random variable; let $P$ be the underlying probability measure; let $\delta > 0$. I wonder if there is already in probability literature a known result giving a sharper bound ...
3
votes
0answers
123 views

Which concentration inequalities apply when moments are infinite?

I have 2 questions: Suppose I have a finite mean but an infinite variance for a discrete distribution w/support $\{1,2,\dots\}$. Is there any probability inequality tighter than Markov in this case? ...
1
vote
0answers
77 views

Hoeffding's inequality for error measures

When Hoeffding's inequality is used to justify that in sample error and out of sample error track each other, the error measure is simple mismatch -- there is no penalty associated with the different ...
1
vote
0answers
91 views

Detecting outliers in binary data using Mahalanobis distance

I have a binary vector $X_i$, $i=1...N$ of independent Bernoulli variables with parameters $p_i, \mu_i = p_i, \sigma_i^2 = p_i(1-p_i)$ (which is known) and I'm looking for some sort of tail bound to ...
0
votes
0answers
29 views

Determining minimum probability for a standardised variable using the Chebyshev inequality [duplicate]

A report on rural water resources states that the nitrate level of wells in a certain groundwater system has a probability distribution whose mean and standard deviation are 5.2, 2.1 ppm, ...
1
vote
1answer
64 views

Concentration for Conditional Random Variable

Consider a conditional random variable \begin{equation} X = \begin{cases} Y & \quad\quad ,X \in A \\ Z & \quad\quad ,X \in A^\complement \end{cases} \end{equation} $Y$ ...
0
votes
1answer
65 views

Inequality of two independent random variables

My question is related to this one but more specific. Inequality on two random variables We have two continuous random variables, $X$ and $Y$. We know that the expected value of both is 0 (or more ...
1
vote
2answers
275 views

Expectation Inequality with indicator function

When I read proof of Chebyshev's inequality, I came across the problem. At first, the proof is : \begin{align*} P(|X-r|\geq k\sigma) &= E(\chi_{|X-\mu| \geq k\sigma}) \\ &= E(\chi_{\left( \...
1
vote
0answers
34 views

property about the standard deviation of 2 r.v.

If $X,Y$ are $\geq 0$ random variables, how to demonstrate that: $$2*Stdev(X) \leq Stdev(X+Y)+ Stdev(X-Y) $$ $Stdev$ represents the usual standard deviation.
2
votes
1answer
120 views

L2 SVM (squared hinge) theory

The linear L2 SVM can be intuitively understood as \begin{equation} \text{minimize } f(\boldsymbol{w}) = \frac{1}{2} \Vert\boldsymbol{w}\Vert^2_2 + C \sum_{i=1}^m \xi_i^2 \tag{1} \end{equation} ...
0
votes
0answers
88 views

Cauchy- Schwarz inequality

In Gaussian random vector,the correlation of two random variables is always between -1 and +1. How to check this fact by application of Cauchy- Schwartz inequality which states that $E(XY)≤(EX^2)^{...
13
votes
1answer
1k views

Oracle Inequality : In basic terms

I'm going through a paper that uses oracle inequality to prove something but I'm unable to understand what it is even trying to do. When I searched online about 'Oracle Inequality', some sources ...
1
vote
1answer
84 views

Inequality on two random variables

This seems like a really straightforward question but I think maybe I lack the vocabulary to search for it correctly. Given two random variables $X$ and $Y$ with known probability distribution ...
3
votes
1answer
495 views

Zero mean unit variance random variables bound on probability

Let $X_1, X_2$ be zero-mean, unit variance Random Variables with correlation coefficient $\rho$ then $$ P(|X_1|\le\epsilon,|X_2|\le\epsilon) \ge 1-\epsilon^{-2}(1+\sqrt{1-\rho^2}) $$ I tried to used ...
6
votes
1answer
88 views

Lower Bound on $E[\frac{1}{X}]$ for positive symmetric distribution

Let $X$ be positive random variable and its distribution is symmetric about its mean value $m$. Then $$ E\left[\frac{1}{X}\right] \geq \frac{1}{m} + \frac{\sigma^2}{m^3}, $$ where $\sigma^2$ is ...
0
votes
0answers
33 views

Is there a method to fit a bound to the plot of an linear inequality?

I have a physical dataset that is bounded by several different processes, and thus the plot takes the form of a linear inequality: I'm specifically interested in studying the upper bound. Is there a ...
2
votes
2answers
380 views

Variance of the reciprocal of a strictly positive random variable

In this post it is stated that due to Jensen's inequality the expected value of the reciprocal of a strictly postive random variable $X$ will satisfy: $$\mathbb{E}\left[\frac{1}{X}\right] \geq \frac{...
1
vote
0answers
43 views

Convergence absolutely in causal time series

I am studying a theorem given in a Time Series Book (Brockwell and Davis, Time Series Theory and Methods, pag 83, proposition 3.1.1) The proposition is the next one: I have the following question: ...
2
votes
1answer
56 views

Why does Jensen's imply this?

Let $F$ be a convex function. If $Y$ and $Z$ are independent random variables and $EZ=0$, then $$EF(Y) = EF(Y+EZ)\leq E(Y+Z).$$ I fail to understand why the last inequality is true. Can someone ...
1
vote
1answer
214 views

Does this Bonferroni styled inequality also hold for characteristic functions?

This is the popular Bonferroni inequality. Does it also hold for characteristic functions of random variables, as in when $P(A_i)$ is replaced by the characteristic function $\chi(A_i)$ and so forth? ...
2
votes
2answers
2k views

Kullback-Leibler divergence lower bound

Are there any (nontrivial) lower bounds on the KL-divergence between two densities? Informally, I am trying to study problems where $f$ is some target density, and I want to show that if $g$ is chosen ...
2
votes
0answers
41 views

Inequality. in Probability

let $ X \sim N(0,1)$ Prove that $P(X\geq c) \leq e^{-ct +{{t^2}\over {2}} }\\ c\geq0, t\in \Bbb{R} $ I have applied chebyscheffs inequality to come to this result: $P(X\geq c) \leq {{1} \over {c^2}} ...
1
vote
0answers
568 views

Relationship between euclidean distance and covariance matrix distances

Consider three random vectors $x, y, z$ taking values in $\mathbb R^n$. They have mean zero, e.g., $E(x_i)=0$, and covariance matrix $\Omega_x$, $\Omega_y$, $\Omega_z$. Assume that the following ...
0
votes
0answers
26 views

If $E[f(X,U)|X]=0$, does $H[f(X,U)]<H[f(X,U)+g(X)]$?

Suppose everything is continuous and smooth, $X,U$ are independent, $H[f(X,U)|X]>0, E[f(X,U)|X]=0$. Then for $g$ s.t. $H[g(X)]>0$, does $$H[f(X,U)]<H[f(X,U)+g(X)]$$ hold? I did some quick ...
2
votes
1answer
79 views

How much better is the best Moment Bound?

I've been looking at Gabor Lugosi's wonderful notes on concentration of measure inequalities. On page 7 of the notes the exercise asks you to show that $$ min_q\mathbb{E}(X^q)t^{-q} \leq inf_{s\geq ...
3
votes
2answers
601 views

Inequality on variance of sum

I want to prove that $$\operatorname{Var}\left(\sum\limits_{i=1}^m{X_i}\right) \leq m\sum\limits_{i=1}^m{\operatorname{Var}(X_i)} \,. \>$$ A too complicated proof is to write $$ a_{ij}=\sqrt {Cov(...
4
votes
1answer
1k views

KL divergence bounds square of L1 norm

In Cover & Thomas, Elements of Information Theory, at the section on Conditional Limit Theorem (11.6), it is proved that the KL divergence bounds the $\cal{L}_1$-norm from above, $\frac{1}{2\ln2}\...
2
votes
0answers
23 views

Boundary of the Durbin-Watson statistic [duplicate]

The Durbin-Watson statistic to detect autocorrelation in the error terms ranges from 0 to 4. Currently, I am working out why it cannot exceed 4 analytically. The lower boundary case is obvious ...