Questions tagged [probability-inequalities]

Probability Inequalities are useful for bounding quantities that might otherwise be hard to compute. A related concept is a concentration inequality, which specifically provides bounds on how far a random variable deviates from some value.

Filter by
Sorted by
Tagged with
3 votes
1 answer
51 views

Sum of i.i.d. random variables for which Chebyshev inequalities are tight

Chebyshev's inequalities: Let $X$ be a random variable with finite expected value $\mu$ and finite non-zero variance $\sigma^{2}$. Then for any real number $\delta > 0$, $$ \Pr[|X - \mu| \geq \...
  • 131
1 vote
1 answer
44 views

Calibrating the probabilities of Ridge Classifier on imbalanced dataset

I have a classification project on an imbalanced dataset (HomeCredit Kaggle dataset) and I have chosen Ridge Classifier (sklearn's implementation) as the most efficient both in terms of time and in ...
  • 35
0 votes
0 answers
20 views

Bounded in Probability

Consider any random variable $X$ with cdf $F_X(x)$. Then given $\epsilon > 0$, we can bound $X$ in the following way. Because the lower $limit$ of $F_X$ is $0$ and its upper $limit$ is $1$, we can ...
2 votes
1 answer
126 views

Proof of corollary of Hoeffding's inequality

I need to proof a corollary of Hoeffding's inequality, and since I'm not used to doing proofs I really don't know where to begin. Hoeffding's inequality: Let $X_1,...,X_n$ be independent real-valued ...
2 votes
0 answers
45 views

Asymptotic behaviour of product of normal r.v.s

Let $X \sim N(\mu ,1)$ and $Y \sim N(\mu, 1)$ where we have $\mu >0.$ I'm trying to evaluate asymptotically the tail distribution function of product of these two random variables. Let $x>0$, ...
  • 21
0 votes
0 answers
21 views

Inequality of $f(\mathrm{Mode}[X])$ and $\mathrm{Mode}[f(X)]$?

Say I knew the distribution of a continuous variable $X$. If somebody randomly picked out a single instance $x$ and asked me to bet on its value, I would go with the mode of the known distribution of $...
5 votes
1 answer
77 views

Equality in Gaussian Poincare Inequality

The Gaussian Poincare inequality states that: for $f: \mathbb{R}^n \to \mathbb{R}$ and $Z\sim \mathcal{N}(0,I)$, we have that \begin{align} Var(f(Z)) \le E[ \| \nabla f(Z)\|^2]. \end{align} My ...
  • 203
0 votes
0 answers
18 views

Using Chernoff bound as an approximation to upper binomial tail

I'm curious about how well the Chernoff bound approximates the value of the upper tail of a binomial distribution. It is well known that, for $X\sim B(n,p),\ \delta >0$: $$P(X \geq (1+\delta)np) \...
0 votes
1 answer
21 views

Error in applying Chebyshev's inequality

I'm trying to solve a problem using Chebychev^' s Inequality: "Suppose that X is a random variable with mean and variance both equal to 20. What can be said about P(0<X<40)?" P(|X-μ|≥...
1 vote
3 answers
119 views

Prove or disprove : $P[A|B] = P[B]$, the A and B are independent? Is this right?

SOrry if this is extremely easy. I did the following but I'm a little bit unsure about it: Let $A=B$, and $P[A]>0$. Then $$P[A|A] = P[A]$$ But A is not independent with itself: $$P[AA] = P[A] \neq ...
0 votes
0 answers
25 views

What does "tensorization" mean in the entropy tensorization inequality?

I am reading high dimensional statistics written by Wainwright. In chapter 3.1.4 tensorization of entropy is used to extend the entropy bound for univariate functions to multivariate cases. As far as ...
0 votes
0 answers
16 views

Limit of absolute value of expected value of the sequence of random variables under L^p convergence

I want to prove this property. If $X_n \longrightarrow X \: in \: \mathbb{L}^p(\Omega)$, then $E[|X_n|^p] \longrightarrow E[|X|^p]$ I tried many ways to do it, including using Minkowski's inequality ...
  • 259
2 votes
1 answer
110 views

Why do we need Jensen inequality for variational autoencoders?

Just to clarify, I think I understand all the derivations in context of VAEs pretty well; however, there is one last thing that I need explained. There are multiple related derivations of the evidence ...
1 vote
1 answer
59 views

Solving for the parameter of an exponential distribution

Suppose I have a random variable $X$ where $X$ follows an exponential distribution of the following form: $$f_X(x) = \frac{1}{\lambda}e^{-\frac{x}{\lambda}}$$. I want to find the value of $\lambda$ ...
0 votes
0 answers
15 views

Product of a series and asequence of random variables

Suppose $\{a_n\}_{n \in N} $ be a bounded sequence positive numberse bounded by $b$ and $Z_j,j=1,2,3,...$ be a sequnce of complex random variables such that $E(|z_j|)<\frac{1}{\sqrt{(j+1)m}}$ for ...
1 vote
0 answers
35 views

For $Y \geq 0$, prove that $Pr(Y \geq k) \leq E(Y)/k$

Let $Y$ be a non-negative random variable, $k$ be any positive constant, show that $Pr(Y \geq k) \leq E(Y)/k$. My attempt (using integration by parts): \begin{align} \int_0^k y \,dF(y) &\leq E(Y) \...
  • 2,213
6 votes
2 answers
89 views

How to prove this inequality?

For any nonnegative random variable $X$ independent of $U$ where $U \sim \operatorname{Uniform}(-t,t)$ and any $t\ge 0,$ $$P(X+U\ge t)\le\frac{E(X)}{2t}.$$ Any hints to prove this inequality?
1 vote
1 answer
103 views

Probability Conditioned on Inequality

Assume that $A \sim \mathcal{N}(0, 1)$, $B \sim \mathcal{N}(0, 1)$. I am trying to calculate $P(A \,|\, A < B)$. For the sake of this problem, we can assume that $A \perp B$, but (for obvious ...
2 votes
1 answer
110 views

Is my way of deriving a statistical test from Hoeffding's inequality correct?

I'm trying to deduce from samples of observations from two finite sets of random variables $X_{1}, ..., X_{n}$ and $Y_{1}, ..., Y_{m}$ that the expected values of the average of those random variables ...
  • 155
2 votes
1 answer
77 views

symmetrization in glivenko-cantelli proof

In this proof of the Glivenko-Cantelli theorem, page 2 of these notes, two types of symmetrization are used. The first transforms the sup of the centered empirical cdf $$P(\sup_{z\in\mathbb{R}}|(1/n)\...
1 vote
0 answers
51 views

Khintchine inequality for the linear combination of sparse Bernoulli random variables

Let $\{\epsilon_{n}\}_{n=1}^{N}$ be i.i.d. random variables with $P(\epsilon_{n} = \pm 1) = 1/2$ for $n=1,2, \ldots, N$ i.e. a sequence of Rademacher distribution. Let $0<p<\infty$ and let $x_{1}...
  • 179
0 votes
0 answers
34 views

Training and worst-case errors

After training a neural network with $N$ input-output pairs $\{(x_i, y_i)\}_{i = 1}^N = \{(x_i, f(x_i))\}_{i = 1}^N$ ($x_i$ are randomly sampled in some set $\mathcal{X}$), one gets the maximum error ...
4 votes
1 answer
82 views

Multivariate Chebyshev's inequality with Mahalanobis distance

In Chebyshev's inequality, we can generalize the 68-95-99.7 rule from normal distributions to bound how much density is within a certain number of standard deviations from the mean. $$ P\big( \big\...
  • 35.8k
1 vote
0 answers
46 views

Probability inequality

Let $X_1,X_2,...X_n$ are independent random variables such that $P(X_i=v_i)=1/v_i$ and $P(X_i=0)=1-1/v_i$, where $1\leq v_i\leq n+1$ for $i=1,2,...,n$. I want to prove that $P(X_1+X_2+...+X_n\geq n+1)\...
  • 21
2 votes
1 answer
129 views

Proof utilizing Chebyshev's inequality

I'm being asked to show that $P(|X-\mu|\geq t) \leq \beta_4/t^4,$ where $\beta_4=E((X-\mu)^4)$. I'm familiar with Chebyshev's Inequality, which similarly states that $P(|X-\mu|\geq t) \leq \sigma^2/t^...
  • 61
0 votes
0 answers
8 views

Variance inequality for nested sets

Let $X, Y,$ and $Z$ are three random variables/vectors, and let $f(., ., .)$ is a real-valued, deterministic function. If $Z$ is independent of $\{X, Y\}$ (e.g., $X, Y, Z$ are independent) then \begin{...
  • 799
0 votes
0 answers
14 views

Test for mean 0 based on Bennett inequality?

Bennett's inequality provides a bound on the probability for the sum of bounded mean 0 random variables to exceed a specified value. It makes no assumptions about distribution, and is stronger than ...
2 votes
1 answer
134 views

Independent Bernoulli random variables

My professor left us to solve this problem: Let $\xi_1, \xi_2,...,\xi_n$ be independent Bernoulli random variables defined on a probability space $(\Omega,P(\Omega),\mathbb{P})$ such that: $\mathbb{P}(...
  • 119
2 votes
0 answers
20 views

Does Cramer's condition imply strong mixing?

In Theorem 1.4 of D. Bosq the Cramer's condition is a prerequisite for the tail bound of sum of dependent variables. The Theorem is as follows: Let $(X_t,t\in\mathbb{Z})$ be a zero-mean real-valued ...
  • 1,072
0 votes
0 answers
61 views

Does Chebyshev's inequality sacrifice a little power for simplicity?

I followed the proof in Chapter 2 of Ross Introduction to probability and statistics for Engineers. As follows; Chebyshev inequality For any K > 1 ( for K< 1 still holds for 0<K_1 but trivial ...
  • 439
3 votes
0 answers
85 views

What are the arguments in getting from the theorem of Vapnik & Chervonenkis (1971) to the common form seen in Devroye, Györfi & Lugosi (1996)?

Context. The theorem below is attributed to "Vapnik, V., and Chervonenkis, A. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability ...
  • 2,236
0 votes
0 answers
97 views

Markov's Inequality and convergence in probability

Theorem: (Markov's Inequality): Let x be a non-negative random variable. Then, for all $b >0$ $P[x \geq b] \leq \frac{E(x)}{b}$. ################# Suppose that $ x_{t}$ is a non-negative random ...
4 votes
1 answer
138 views

Corollary of Hoeffding’s Inequality

Question I am not from a statistics background. I came across the following corollary of Hoeffding’s Inequality and couldn't find the derivation or proof for it. Could anyone please share some ...
2 votes
0 answers
77 views

Problem understanding the intuition behind Slepian's inequality

Slepian's inequality is defined as follows: Let $X\in\mathbb{R}^n$ and $Y\in\mathbb{R}^n$ be centered Gaussian random vectors such that \begin{align} \mathbb{E}X_iX_j&\geq \mathbb{E}Y_iY_j,\quad \...
  • 1,072
1 vote
1 answer
210 views

Markov Inequality for Sum

I don't have proper knowledge of probability, I was surfing around internet about Markov's inequality, I found a paper on JSTOR titled "The Markov Inequality for Sums of Independent Random ...
6 votes
2 answers
153 views

Bounding sum of quartic deviations from sample mean

[Cross-posted here with no answers for a few days] I came - to the very best of my knowledge from reading the source - across the following statement in The Jackknife and Bootstrap, Shao and Tu, p. 87:...
4 votes
1 answer
197 views

Hoeffding type concentration result for the inverse of a sum of iid random variables

Consider a collection of $n$ i.i.d. Bernoulli random variables $\{ X_i \}_{i=1}^{n}$ with $\mathbb{E}[X_i] = \mu$. Then, if $\hat{\mu}$ is the mean of the $n$ random variables, i.e. if, $$\hat{\mu} = \...
  • 155
1 vote
0 answers
66 views

Holder's inequality in the case of $L_1$ and $L_{\infty}$ norm

I am referring to Wainwright's High-Dimensional Statistics book, where at some point it is deduced that \begin{equation} \frac{w'X\Delta}{n}\leq \left\lVert\frac{w'X}{n}\right\rVert_{\infty}\lVert\...
  • 1,072
1 vote
1 answer
151 views

Proving Chebyshev's Inequality

I'm working on proving Chebyshev's Inequality. I watched this YouTube video and it almost makes sense. There is one step in the proof I don't understand. Using Markov's Inequality you substitute ...
1 vote
0 answers
87 views

Can we use Frechet inequalities to infer significance of collections of hypothesis tests?

There are numerous issues that have been identified both in the theory and practice of $p$-values, including the arbitrariness of confidence levels, interpretation, and tail-risk in the ...
  • 5,102
1 vote
1 answer
76 views

Minimum number of samples outside interval, unknown distribution

I'm considering a graph that shows the mean of $n$ samples drawn from an unknown, continuous population. It also shows the standard error of the means. From this I calculated the standard deviation $\...
  • 123
2 votes
0 answers
54 views

Concrete bound of expected value of a difference of I.I.D. Uniform Random Variables

In the following, $X,X_1,X_2,\dots X_n$ are I.I.D. uniform random variables in $[0,1]^d$ in $\mathbb{R}^d$. The problem I am attempting to solve is Exercise 2.4 from Gyorfi's "A distribution free ...
2 votes
2 answers
423 views

How to derive Paley-Zigmund Inequality proof

The Paley-Zygmund inequality is given by \begin{equation} \operatorname{P}( Z > \theta\operatorname{E}[Z] ) \ge (1-\theta)^2 \frac{\operatorname{E}[Z]^2}{\operatorname{E}[Z^2]} \end{equation} I ...
1 vote
1 answer
91 views

Finding a consistent sequence of estimators such that $\lim_{n\to\infty} E_\theta[(W_n-\theta)^2]\ne 0$

There are many ways to check if a sequence of estimators is consistent. By definition, a sequence of estimators $W_n = W_n(X_1,X_2,\ldots,X_n)$ is a consistent sequence of estimators of the parameter ...
2 votes
0 answers
55 views

0-1 laws in random graphs: probability $\beta$ is large if $k$ is large

How has the author derived here on the page 3 in the context of random graphs and 0-1 laws that $\beta$ is large if $$k\geq ((\frac{2}{\alpha})\log n)^{\frac{1}{2}}$$ ? What I did is this: I've ...
3 votes
1 answer
29 views

monotonicity of sample averages tails as a function of sample size

Let $X_1,...$ be iid mean zero random variables. The LLN says $\overline{X}_n\to 0$. I am curious if the following is true: is $P(|\overline{X}_n|>x)>P(|\overline{X}_{n+1}|>x)$ for $x$ ...
2 votes
1 answer
37 views

Show $(E|X|^2)/(E|X^2|) \leq P(X \not =0)$

I'm looking to show this inequality is true, and in turn use it to conclude the second moment method's bound. Show that $\frac{E|X|^2}{E|X^2|} \leq P(X \not =0)$. Again, I'm not supposed to use second ...
  • 145
4 votes
2 answers
145 views

Does maximizing Jensen–Shannon divergence maximize Kullback–Leibler divergence?

Does maximizing the Jensen–Shannon divergence $D_{\mathrm{JS}}(P \parallel Q)$ maximize the Kullback–Leibler divergence $D_{\mathrm{KL}}(P \parallel Q)$? If so, I'd like to be able to show that it ...
1 vote
0 answers
107 views

Upper Bound for 2nd Raw Moment of Positive Random Variable

Let $X$ be a random variable with support $(0,\infty)$. All I know about $X$ is the support, finite higher moments, and $\mathbb{E}(X)=\mu$. I am trying to come up with a more tractable upper bound ...
4 votes
1 answer
84 views

Prove that $E[\log(\alpha X_t^2)] < 0 $ implies $\alpha < 3.5622$ with $X_t \sim N(0,1)$

I am trying to prove this statement: If $X_t \sim N(0,1)$ then $$E[\log(\alpha X_t^2)] < 0 \implies \alpha < 3.5622$$ which is a a necessary condition often found in textbooks for strict ...
  • 137

1
2 3 4 5
7