Questions tagged [inequality]

Use this tag if you question involves the use of an inequality. The inequality may have probabilistic origins or be a purely mathematical inequality. Do not use for measures of inequality, for instance income inequality. For that use [tag:diversity].

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

How do I solve a linear inequality system ($X\beta+b<0$)?

Given a low-dimension linear regression problem $\mathbf{y}=\mathbf{X}\beta + \epsilon$, we can easily estimate $\beta$ with $(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^Ty$. However, the problem seems ...
1
vote
0answers
56 views

Some inequality problem, which is larger?

There is a set of n random numbers whose sum equals 1, i.e., $$w_a=\{w_{a1},\; w_{a2},\; \ldots,\; w_{an}\}$$ where $$ w_{a1} + w_{a2}+ \ldots+ w_{an} =1$$ and $$0 < \{ w_{a1}, w_{a2}, \ldots, w_{...
3
votes
1answer
55 views

Need help to understand Feller's statement “whenever $r$th moment exists so do all preceding moments”

I am reading the book of Feller called "An Introduction to Probability Theory and Its Applications, Vol I" (third edition, page 227) and am stuck at the moment he explains the notion of variance of a ...
0
votes
0answers
17 views

Conditioning and linear MSE

Let $\sigma_{X|Y}^2$ denote the linear mean squared error in estimating $X$ from $Y$. Then is it always true that additional conditioning cannot increase the LLSE? In other words, is this true? $$ \...
1
vote
1answer
37 views

Moment inequality: $E\mid X_1 X_2 X_3\mid \leq (E(\mid X_1\mid^3)+E(\mid X_2\mid^3)+E(\mid X_3\mid^3))/3 $ for zero-mean r.v.'s?

Let $X_1, X_2, X_3$ be zero mean random variables and assume $E(\mid X_i \mid ^{4+\delta})\leq C, i=1,2,3$ where $C$ is a constant and $\delta>0$ some positive small constant. How can I show that ...
1
vote
1answer
26 views

Unusual Markov inequality for normal distribution

I'm trying to answer the following question from Larry Wassermans book on statistical inference. My question is how did they arrive at the Markov bound, it does not seem like the normal form of the ...
0
votes
1answer
43 views

Variance and covariance inequality

Given a real-valued random variable $X$, is $$2\mathbb E[X] \mathrm{Var}(X) \geq \mathrm{Cov}(X, X^2)$$ true? Any pointers for how to tackle this problem would be immensely helpful.
5
votes
1answer
67 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
2answers
170 views

Proving efficiency of OLS over GLS

I'm trying to prove the efficiency of OLS over GLS when the covariance matrix of the error $\varepsilon$ is mistakenly assumed to be $\sigma^2\Sigma$ instead of $\sigma^2 I$. After deriving the ...
3
votes
0answers
33 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
0answers
34 views

Can I test for inequality in H0 using chi square test?

Let's say I want to test whether an $n$-sided dice is not too unfair. In the standard chi-square test we test the zero-hypothesis $$ H_0\colon (p_1,\dots p_n) = (1/n,\dots,1/n) ,\quad\text{i.e.,}\quad ...
2
votes
1answer
32 views

Estimator based on inequality data

$X_i \sim N(\mu, \sigma^2)$ (iid), $i = 1,2,...,N$, I want to estimate $\theta = (\mu, \sigma^2)$. Problem is, I don't observe $x_i$. For each $i$, I only observe $(a_i, b_i)$, and I know that $a_i &...
2
votes
2answers
38 views

Does an inequality hold as an expectation over a probability distribution?

Suppose I have to functions $f(x)$ and $g(x)$ such that $$ f(x) \leq g(x) \quad \forall x. $$ For a distribution $\pi(x)$ on $x$, is it necessarily true that $$ E_\pi[f(x)] \leq E_\pi[g(x)]? $$ My ...
0
votes
1answer
100 views

A different proof for KL divergence non-negativity

KL divergence's non-negativity can be proved in many ways. One could use the inequality $\log x \leq x - 1$ as a main step in the proof, another one could leverage the property of concave of the ...
2
votes
3answers
78 views

Cauchy Schwarz inequality proof using discriminant

I know the proof but I'm unclear on one thing. Cauchy-Schwarz inequality: Given X,Y are random variables, the following holds: $$ (E[XY])^2 \le E[X^2]E[Y^2] $$ Proof Let $$ u(t) = E[(tX - Y)^2] $$ ...
4
votes
0answers
65 views

Proving an inequality for CDF's

I am working on a proof to show that given $x_1, x_2,\ldots,x_k$ random variables with a joint pdf and joint CDF, show that $$ 1-\sum_{i=1}^k \overline{F_i(x_i)} \leq F(x_1,x_2,\ldots,x_k) \leq \min_i ...
1
vote
1answer
26 views

Help understanding a probability inequality

I'm working throught Wasserman's "All of Statistics" book. When proving convergence of random variables/distributions in chapter 5, he lists the following inequality: $$F_n(x) = \mathbb{P}(X_n\le x)=\...
0
votes
1answer
139 views

Prove that $E(X\ln X)\le E X E\ln X$ [closed]

I want to prove it using Jensen inequality, so I need to prove that $g(x)=x\ln x$ is a convex function, which means $$g\left(\frac{a+b}{2}\right)\le \frac{1}{2}\left(g(a)+g(b)\right).$$ How can I ...
1
vote
1answer
36 views

How does Chebyshev's inequality imply $P(X ≥ k) ≤ 1/(σk)^2$?

I am aware that Chebyshev's inequality $P(X ≥ kσ) ≤ 1/k^2$ can also be written as $P(X ≥ k) ≤ 1/(σk)^2$, but I do not understand the math to convert between these forms. Could someone explain/point me ...
2
votes
1answer
113 views

An inequality giving a sharper bound than that given by the Chebyshev's?

Let $X > 0$ be a random variable; let $P$ be the underlying probability measure; let $\delta > 0$. I wonder if there is already in probability literature a known result giving a sharper bound ...
3
votes
0answers
108 views

Which concentration inequalities apply when moments are infinite?

I have 2 questions: Suppose I have a finite mean but an infinite variance for a discrete distribution w/support $\{1,2,\dots\}$. Is there any probability inequality tighter than Markov in this case? ...
1
vote
0answers
67 views

Hoeffding's inequality for error measures

When Hoeffding's inequality is used to justify that in sample error and out of sample error track each other, the error measure is simple mismatch -- there is no penalty associated with the different ...
1
vote
0answers
86 views

Detecting outliers in binary data using Mahalanobis distance

I have a binary vector $X_i$, $i=1...N$ of independent Bernoulli variables with parameters $p_i, \mu_i = p_i, \sigma_i^2 = p_i(1-p_i)$ (which is known) and I'm looking for some sort of tail bound to ...
0
votes
0answers
25 views

Determining minimum probability for a standardised variable using the Chebyshev inequality [duplicate]

A report on rural water resources states that the nitrate level of wells in a certain groundwater system has a probability distribution whose mean and standard deviation are 5.2, 2.1 ppm, ...
1
vote
1answer
56 views

Concentration for Conditional Random Variable

Consider a conditional random variable \begin{equation} X = \begin{cases} Y & \quad\quad ,X \in A \\ Z & \quad\quad ,X \in A^\complement \end{cases} \end{equation} $Y$ ...
0
votes
1answer
56 views

Inequality of two independent random variables

My question is related to this one but more specific. Inequality on two random variables We have two continuous random variables, $X$ and $Y$. We know that the expected value of both is 0 (or more ...
0
votes
0answers
78 views

Inequality Expectation of Truncated Random Variable

I found this inequality which looks cute but could not find the proof. Let $X$ be a random variable and $c$ a positive constant. Show that for $p>1$ $$ E ( | X \, {\bf 1}\{|X|>c\} | ) \leq { ...
1
vote
2answers
227 views

Expectation Inequality with indicator function

When I read proof of Chebyshev's inequality, I came across the problem. At first, the proof is : \begin{align*} P(|X-r|\geq k\sigma) &= E(\chi_{|X-\mu| \geq k\sigma}) \\ &= E(\chi_{\left( \...
1
vote
0answers
33 views

property about the standard deviation of 2 r.v.

If $X,Y$ are $\geq 0$ random variables, how to demonstrate that: $$2*Stdev(X) \leq Stdev(X+Y)+ Stdev(X-Y) $$ $Stdev$ represents the usual standard deviation.
2
votes
1answer
103 views

L2 SVM (squared hinge) theory

The linear L2 SVM can be intuitively understood as \begin{equation} \text{minimize } f(\boldsymbol{w}) = \frac{1}{2} \Vert\boldsymbol{w}\Vert^2_2 + C \sum_{i=1}^m \xi_i^2 \tag{1} \end{equation} ...
0
votes
0answers
79 views

Cauchy- Schwarz inequality

In Gaussian random vector,the correlation of two random variables is always between -1 and +1. How to check this fact by application of Cauchy- Schwartz inequality which states that $E(XY)≤(EX^2)^{...
12
votes
1answer
1k views

Oracle Inequality : In basic terms

I'm going through a paper that uses oracle inequality to prove something but I'm unable to understand what it is even trying to do. When I searched online about 'Oracle Inequality', some sources ...
1
vote
1answer
67 views

Inequality on two random variables

This seems like a really straightforward question but I think maybe I lack the vocabulary to search for it correctly. Given two random variables $X$ and $Y$ with known probability distribution ...
3
votes
1answer
470 views

Zero mean unit variance random variables bound on probability

Let $X_1, X_2$ be zero-mean, unit variance Random Variables with correlation coefficient $\rho$ then $$ P(|X_1|\le\epsilon,|X_2|\le\epsilon) \ge 1-\epsilon^{-2}(1+\sqrt{1-\rho^2}) $$ I tried to used ...
6
votes
1answer
84 views

Lower Bound on $E[\frac{1}{X}]$ for positive symmetric distribution

Let $X$ be positive random variable and its distribution is symmetric about its mean value $m$. Then $$ E\left[\frac{1}{X}\right] \geq \frac{1}{m} + \frac{\sigma^2}{m^3}, $$ where $\sigma^2$ is ...
0
votes
0answers
33 views

Is there a method to fit a bound to the plot of an linear inequality?

I have a physical dataset that is bounded by several different processes, and thus the plot takes the form of a linear inequality: I'm specifically interested in studying the upper bound. Is there a ...
2
votes
2answers
336 views

Variance of the reciprocal of a strictly positive random variable

In this post it is stated that due to Jensen's inequality the expected value of the reciprocal of a strictly postive random variable $X$ will satisfy: $$\mathbb{E}\left[\frac{1}{X}\right] \geq \frac{...
1
vote
0answers
39 views

Convergence absolutely in causal time series

I am studying a theorem given in a Time Series Book (Brockwell and Davis, Time Series Theory and Methods, pag 83, proposition 3.1.1) The proposition is the next one: I have the following question: ...
2
votes
1answer
53 views

Why does Jensen's imply this?

Let $F$ be a convex function. If $Y$ and $Z$ are independent random variables and $EZ=0$, then $$EF(Y) = EF(Y+EZ)\leq E(Y+Z).$$ I fail to understand why the last inequality is true. Can someone ...
1
vote
1answer
204 views

Does this Bonferroni styled inequality also hold for characteristic functions?

This is the popular Bonferroni inequality. Does it also hold for characteristic functions of random variables, as in when $P(A_i)$ is replaced by the characteristic function $\chi(A_i)$ and so forth? ...
2
votes
2answers
1k views

Kullback-Leibler divergence lower bound

Are there any (nontrivial) lower bounds on the KL-divergence between two densities? Informally, I am trying to study problems where $f$ is some target density, and I want to show that if $g$ is chosen ...
2
votes
0answers
40 views

Inequality. in Probability

let $ X \sim N(0,1)$ Prove that $P(X\geq c) \leq e^{-ct +{{t^2}\over {2}} }\\ c\geq0, t\in \Bbb{R} $ I have applied chebyscheffs inequality to come to this result: $P(X\geq c) \leq {{1} \over {c^2}} ...
1
vote
0answers
515 views

Relationship between euclidean distance and covariance matrix distances

Consider three random vectors $x, y, z$ taking values in $\mathbb R^n$. They have mean zero, e.g., $E(x_i)=0$, and covariance matrix $\Omega_x$, $\Omega_y$, $\Omega_z$. Assume that the following ...
0
votes
0answers
26 views

If $E[f(X,U)|X]=0$, does $H[f(X,U)]<H[f(X,U)+g(X)]$?

Suppose everything is continuous and smooth, $X,U$ are independent, $H[f(X,U)|X]>0, E[f(X,U)|X]=0$. Then for $g$ s.t. $H[g(X)]>0$, does $$H[f(X,U)]<H[f(X,U)+g(X)]$$ hold? I did some quick ...
2
votes
1answer
76 views

How much better is the best Moment Bound?

I've been looking at Gabor Lugosi's wonderful notes on concentration of measure inequalities. On page 7 of the notes the exercise asks you to show that $$ min_q\mathbb{E}(X^q)t^{-q} \leq inf_{s\geq ...
1
vote
2answers
501 views

Inequality on variance of sum

I want to prove that $$\operatorname{Var}\left(\sum\limits_{i=1}^m{X_i}\right) \leq m\sum\limits_{i=1}^m{\operatorname{Var}(X_i)} \,. \>$$ A too complicated proof is to write $$ a_{ij}=\sqrt {Cov(...
4
votes
1answer
1k views

KL divergence bounds square of L1 norm

In Cover & Thomas, Elements of Information Theory, at the section on Conditional Limit Theorem (11.6), it is proved that the KL divergence bounds the $\cal{L}_1$-norm from above, $\frac{1}{2\ln2}\...
2
votes
0answers
23 views

Boundary of the Durbin-Watson statistic [duplicate]

The Durbin-Watson statistic to detect autocorrelation in the error terms ranges from 0 to 4. Currently, I am working out why it cannot exceed 4 analytically. The lower boundary case is obvious ...
0
votes
1answer
19 views

How can you map the exceedance of a threshold into an activation function of a Neural Network?

I am totally new to Artificial Neural Networks. Let’s say that the model you are trying to turn into an artificial neural network has an output that is triggered only by the exceedance of a threshold: ...
1
vote
0answers
178 views

Bounding the distance between the mean and the mode of a unimodal distribution

It can be derived from here by the triangle inequality that for unimodal distributions, the distance between the mean and the mode satisfies the bound $$\left|\overline{X} - \text{mode}(X) \right| \...