Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [inequality]

Use this tag if you question involves the use of an inequality. The inequality may have probabilistic origins or be a purely mathematical inequality. Do not use for measures of inequality, for instance income inequality. For that use [tag:diversity].

0
votes
1answer
38 views

Variance and covariance inequality

Given a real-valued random variable $X$, is $$2\mathbb E[X] \mathrm{Var}(X) \geq \mathrm{Cov}(X, X^2)$$ true? Any pointers for how to tackle this problem would be immensely helpful.
5
votes
1answer
63 views

Bounding residual variance with distance from mean

For a linear regression $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0,\sigma^2 I)$, we have $\hat Y = H Y$ for $H = X(X^TX)^{-1}X^T$. This means that $Var(Y - \hat Y) = \sigma^2(I-H)$ ...
1
vote
2answers
139 views

Proving efficiency of OLS over GLS

I'm trying to prove the efficiency of OLS over GLS when the covariance matrix of the error $\varepsilon$ is mistakenly assumed to be $\sigma^2\Sigma$ instead of $\sigma^2 I$. After deriving the ...
3
votes
0answers
26 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
0answers
34 views

Can I test for inequality in H0 using chi square test?

Let's say I want to test whether an $n$-sided dice is not too unfair. In the standard chi-square test we test the zero-hypothesis $$ H_0\colon (p_1,\dots p_n) = (1/n,\dots,1/n) ,\quad\text{i.e.,}\quad ...
2
votes
1answer
32 views

Estimator based on inequality data

$X_i \sim N(\mu, \sigma^2)$ (iid), $i = 1,2,...,N$, I want to estimate $\theta = (\mu, \sigma^2)$. Problem is, I don't observe $x_i$. For each $i$, I only observe $(a_i, b_i)$, and I know that $a_i &...
2
votes
2answers
38 views

Does an inequality hold as an expectation over a probability distribution?

Suppose I have to functions $f(x)$ and $g(x)$ such that $$ f(x) \leq g(x) \quad \forall x. $$ For a distribution $\pi(x)$ on $x$, is it necessarily true that $$ E_\pi[f(x)] \leq E_\pi[g(x)]? $$ My ...
0
votes
1answer
80 views

A different proof for KL divergence non-negativity

KL divergence's non-negativity can be proved in many ways. One could use the inequality $\log x \leq x - 1$ as a main step in the proof, another one could leverage the property of concave of the ...
2
votes
3answers
73 views

Cauchy Schwarz inequality proof using discriminant

I know the proof but I'm unclear on one thing. Cauchy-Schwarz inequality: Given X,Y are random variables, the following holds: $$ (E[XY])^2 \le E[X^2]E[Y^2] $$ Proof Let $$ u(t) = E[(tX - Y)^2] $$ ...
4
votes
0answers
64 views

Proving an inequality for CDF's

I am working on a proof to show that given $x_1, x_2,\ldots,x_k$ random variables with a joint pdf and joint CDF, show that $$ 1-\sum_{i=1}^k \overline{F_i(x_i)} \leq F(x_1,x_2,\ldots,x_k) \leq \min_i ...
1
vote
1answer
25 views

Help understanding a probability inequality

I'm working throught Wasserman's "All of Statistics" book. When proving convergence of random variables/distributions in chapter 5, he lists the following inequality: $$F_n(x) = \mathbb{P}(X_n\le x)=\...
0
votes
1answer
126 views

Prove that $E(X\ln X)\le E X E\ln X$ [closed]

I want to prove it using Jensen inequality, so I need to prove that $g(x)=x\ln x$ is a convex function, which means $$g\left(\frac{a+b}{2}\right)\le \frac{1}{2}\left(g(a)+g(b)\right).$$ How can I ...
1
vote
1answer
36 views

How does Chebyshev's inequality imply $P(X ≥ k) ≤ 1/(σk)^2$?

I am aware that Chebyshev's inequality $P(X ≥ kσ) ≤ 1/k^2$ can also be written as $P(X ≥ k) ≤ 1/(σk)^2$, but I do not understand the math to convert between these forms. Could someone explain/point me ...
2
votes
1answer
96 views

An inequality giving a sharper bound than that given by the Chebyshev's?

Let $X > 0$ be a random variable; let $P$ be the underlying probability measure; let $\delta > 0$. I wonder if there is already in probability literature a known result giving a sharper bound ...
3
votes
0answers
104 views

Which concentration inequalities apply when moments are infinite?

I have 2 questions: Suppose I have a finite mean but an infinite variance for a discrete distribution w/support $\{1,2,\dots\}$. Is there any probability inequality tighter than Markov in this case? ...
1
vote
0answers
66 views

Hoeffding's inequality for error measures

When Hoeffding's inequality is used to justify that in sample error and out of sample error track each other, the error measure is simple mismatch -- there is no penalty associated with the different ...
1
vote
0answers
79 views

Detecting outliers in binary data using Mahalanobis distance

I have a binary vector $X_i$, $i=1...N$ of independent Bernoulli variables with parameters $p_i, \mu_i = p_i, \sigma_i^2 = p_i(1-p_i)$ (which is known) and I'm looking for some sort of tail bound to ...
0
votes
0answers
22 views

Determining minimum probability for a standardised variable using the Chebyshev inequality [duplicate]

A report on rural water resources states that the nitrate level of wells in a certain groundwater system has a probability distribution whose mean and standard deviation are 5.2, 2.1 ppm, ...
1
vote
1answer
50 views

Concentration for Conditional Random Variable

Consider a conditional random variable \begin{equation} X = \begin{cases} Y & \quad\quad ,X \in A \\ Z & \quad\quad ,X \in A^\complement \end{cases} \end{equation} $Y$ ...
0
votes
1answer
54 views

Inequality of two independent random variables

My question is related to this one but more specific. Inequality on two random variables We have two continuous random variables, $X$ and $Y$. We know that the expected value of both is 0 (or more ...
0
votes
0answers
72 views

Inequality Expectation of Truncated Random Variable

I found this inequality which looks cute but could not find the proof. Let $X$ be a random variable and $c$ a positive constant. Show that for $p>1$ $$ E ( | X \, {\bf 1}\{|X|>c\} | ) \leq { ...
1
vote
2answers
202 views

Expectation Inequality with indicator function

When I read proof of Chebyshev's inequality, I came across the problem. At first, the proof is : \begin{align*} P(|X-r|\geq k\sigma) &= E(\chi_{|X-\mu| \geq k\sigma}) \\ &= E(\chi_{\left( \...
1
vote
0answers
33 views

property about the standard deviation of 2 r.v.

If $X,Y$ are $\geq 0$ random variables, how to demonstrate that: $$2*Stdev(X) \leq Stdev(X+Y)+ Stdev(X-Y) $$ $Stdev$ represents the usual standard deviation.
2
votes
1answer
94 views

L2 SVM (squared hinge) theory

The linear L2 SVM can be intuitively understood as \begin{equation} \text{minimize } f(\boldsymbol{w}) = \frac{1}{2} \Vert\boldsymbol{w}\Vert^2_2 + C \sum_{i=1}^m \xi_i^2 \tag{1} \end{equation} ...
0
votes
0answers
78 views

Cauchy- Schwarz inequality

In Gaussian random vector,the correlation of two random variables is always between -1 and +1. How to check this fact by application of Cauchy- Schwartz inequality which states that $E(XY)≤(EX^2)^{...
12
votes
1answer
894 views

Oracle Inequality : In basic terms

I'm going through a paper that uses oracle inequality to prove something but I'm unable to understand what it is even trying to do. When I searched online about 'Oracle Inequality', some sources ...
1
vote
1answer
64 views

Inequality on two random variables

This seems like a really straightforward question but I think maybe I lack the vocabulary to search for it correctly. Given two random variables $X$ and $Y$ with known probability distribution ...
3
votes
1answer
455 views

Zero mean unit variance random variables bound on probability

Let $X_1, X_2$ be zero-mean, unit variance Random Variables with correlation coefficient $\rho$ then $$ P(|X_1|\le\epsilon,|X_2|\le\epsilon) \ge 1-\epsilon^{-2}(1+\sqrt{1-\rho^2}) $$ I tried to used ...
6
votes
1answer
82 views

Lower Bound on $E[\frac{1}{X}]$ for positive symmetric distribution

Let $X$ be positive random variable and its distribution is symmetric about its mean value $m$. Then $$ E\left[\frac{1}{X}\right] \geq \frac{1}{m} + \frac{\sigma^2}{m^3}, $$ where $\sigma^2$ is ...
0
votes
0answers
32 views

Is there a method to fit a bound to the plot of an linear inequality?

I have a physical dataset that is bounded by several different processes, and thus the plot takes the form of a linear inequality: I'm specifically interested in studying the upper bound. Is there a ...
2
votes
2answers
310 views

Variance of the reciprocal of a strictly positive random variable

In this post it is stated that due to Jensen's inequality the expected value of the reciprocal of a strictly postive random variable $X$ will satisfy: $$\mathbb{E}\left[\frac{1}{X}\right] \geq \frac{...
1
vote
0answers
37 views

Convergence absolutely in causal time series

I am studying a theorem given in a Time Series Book (Brockwell and Davis, Time Series Theory and Methods, pag 83, proposition 3.1.1) The proposition is the next one: I have the following question: ...
3
votes
1answer
53 views

Why does Jensen's imply this?

Let $F$ be a convex function. If $Y$ and $Z$ are independent random variables and $EZ=0$, then $$EF(Y) = EF(Y+EZ)\leq E(Y+Z).$$ I fail to understand why the last inequality is true. Can someone ...
1
vote
1answer
204 views

Does this Bonferroni styled inequality also hold for characteristic functions?

This is the popular Bonferroni inequality. Does it also hold for characteristic functions of random variables, as in when $P(A_i)$ is replaced by the characteristic function $\chi(A_i)$ and so forth? ...
1
vote
1answer
1k views

Kullback-Leibler divergence lower bound

Are there any (nontrivial) lower bounds on the KL-divergence between two densities? Informally, I am trying to study problems where $f$ is some target density, and I want to show that if $g$ is chosen ...
2
votes
0answers
40 views

Inequality. in Probability

let $ X \sim N(0,1)$ Prove that $P(X\geq c) \leq e^{-ct +{{t^2}\over {2}} }\\ c\geq0, t\in \Bbb{R} $ I have applied chebyscheffs inequality to come to this result: $P(X\geq c) \leq {{1} \over {c^2}} ...
1
vote
0answers
464 views

Relationship between euclidean distance and covariance matrix distances

Consider three random vectors $x, y, z$ taking values in $\mathbb R^n$. They have mean zero, e.g., $E(x_i)=0$, and covariance matrix $\Omega_x$, $\Omega_y$, $\Omega_z$. Assume that the following ...
0
votes
0answers
26 views

If $E[f(X,U)|X]=0$, does $H[f(X,U)]<H[f(X,U)+g(X)]$?

Suppose everything is continuous and smooth, $X,U$ are independent, $H[f(X,U)|X]>0, E[f(X,U)|X]=0$. Then for $g$ s.t. $H[g(X)]>0$, does $$H[f(X,U)]<H[f(X,U)+g(X)]$$ hold? I did some quick ...
2
votes
1answer
75 views

How much better is the best Moment Bound?

I've been looking at Gabor Lugosi's wonderful notes on concentration of measure inequalities. On page 7 of the notes the exercise asks you to show that $$ min_q\mathbb{E}(X^q)t^{-q} \leq inf_{s\geq ...
1
vote
1answer
439 views

Inequality on variance of sum

I want to prove that $$\operatorname{Var}\left(\sum\limits_{i=1}^m{X_i}\right) \leq m\sum\limits_{i=1}^m{\operatorname{Var}(X_i)} \,. \>$$ A too complicated proof is to write $$ a_{ij}=\sqrt {Cov(...
4
votes
1answer
975 views

KL divergence bounds square of L1 norm

In Cover & Thomas, Elements of Information Theory, at the section on Conditional Limit Theorem (11.6), it is proved that the KL divergence bounds the $\cal{L}_1$-norm from above, $\frac{1}{2\ln2}\...
2
votes
0answers
21 views

Boundary of the Durbin-Watson statistic [duplicate]

The Durbin-Watson statistic to detect autocorrelation in the error terms ranges from 0 to 4. Currently, I am working out why it cannot exceed 4 analytically. The lower boundary case is obvious ...
0
votes
1answer
18 views

How can you map the exceedance of a threshold into an activation function of a Neural Network?

I am totally new to Artificial Neural Networks. Let’s say that the model you are trying to turn into an artificial neural network has an output that is triggered only by the exceedance of a threshold: ...
1
vote
0answers
176 views

Bounding the distance between the mean and the mode of a unimodal distribution

It can be derived from here by the triangle inequality that for unimodal distributions, the distance between the mean and the mode satisfies the bound $$\left|\overline{X} - \text{mode}(X) \right| \...
1
vote
0answers
73 views

The best constant in a martingale moment inequality

Suppose we have a stochastic integral of the form $M_{t}=\int_{0}^{t}{H_{u}dW_{u}}$ and we know $M$ is a (true) martingale. It is known that for all $p\geq1$, $$ \mathbb{E}[|M_{t}|^{p}]^{1/p} \leq C(p)...
1
vote
1answer
213 views

Linear regression question on Idempotent matrix and leverage points

I am considering a linear regression model $Y_i = X_i^T \beta + \epsilon_i, i = 1,2,\dots,n$. where $X_i \in \mathbb{R}^p$. $\epsilon_i$'s are independent copies of random error $\epsilon \in \mathbb{...
1
vote
2answers
54 views

Inequality Problem involving exponential expression [closed]

How do I prove the following inequality : $$\frac{2}{\alpha^2} \bigg( e^{\alpha y} - e^{\alpha x} \bigg) + e^{\alpha x} \bigg( x^2 - y^2 \bigg) > 0 \; \;?$$ Here, $\alpha > 0, y < x$. ...
1
vote
0answers
89 views

Link between the $L_1$ distance between CDFs and PDFs?

Let $F:(-\infty,\infty)\rightarrow[0,1]$ and $G:(-\infty,\infty)\rightarrow[0,1]$ be two CDFs with PDFs $f$ and $g$, respectively. Is there a connection/inequality between: $$d_1 = \int_{-\infty}^{\...
2
votes
1answer
292 views

Monotonicity of special case of Kullback-Leibler divergence

I have two discrete distributions $\tau$ and $\rho$ with the same support $\Omega$. I'm considering a weighted mixture of these distributions described by the following function: $$ f(w) = (1-w) \cdot ...
0
votes
1answer
52 views

Two probabilities and an inequality

I found this fun little problem intriguing as it really questions your assumptions. Given two unknown probabilities, $p_1$ and $p_2$, along with the condition $p_1+p_2 \le 1$, what is the best ...