Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [bounds]

The tag has no usage guidance.

1
vote
1answer
19 views

Is there an upper bound on number of logistic regression responses that yield infinite estimates

Suppose a logistic regression problem has N observations of {0, 1} and that there are p parameters. Also assume the design matrix, X, is full rank with p < N. We know that there will be certain ...
0
votes
0answers
21 views

Approximation of the upper bound on the expectation of log sum of exponentials

I am having some trouble replicating the results in Guillaume Bouchard's paper, Efficient Bounds for the Softmax Function and Applications to Approximate Inference in Hybrid Models, where he discusses ...
1
vote
0answers
49 views

Finding Chernoff bounds maximum estimators

I am currently trying to resolve the following exercise about Chernoff bounds: Let $X_{1}, X_{2}, \dots, X_{n}$ be independent, identically distributed (i.i.d) random variables with distribution $N(0,...
0
votes
0answers
15 views

Reference request: Least Squares Concentration Bounds

I've recently become interested in parameter estimation for regression and time-series type models and I often encounter, and indeed need to understand, results using concentration of measure-type ...
1
vote
0answers
31 views

Tight upper bound on the expectation of a concave function

N is a random variable whose sample space is [0,$\infty$). I have an expression in terms of the expectation of this variable and I want to find a tight upper bound on the whole expression. The ...
0
votes
1answer
28 views

Bounded functions when x tends to infinity

Please help me understand the below: The notation $g(x) = O(f(x))$ denotes that $\left|\frac{f(x)}{g(x)}\right|$ is bounded as $x \to \infty$. For instance if $g(x)=3x^2 + 2$, then $g(x) = O(x^2)$ ...
0
votes
1answer
47 views

Find bayesian credible Interval

I am taking this example from here. They have given the steps but i cannot understand them so a little dumbing down of answer is necessary or you can just explain the answer given there. its on page 5 ...
1
vote
0answers
65 views

Is it true that normalizing the output of a ReLu feedforward Neural Network that its Rademacher Complexity becomes a constant?

I was trying to understand what happened with the Rademacher Complexity: $$ R_S(F) = \frac{1}{m} \mathbb E_{\sigma} [\sup_{f \in F}\sum^m_{i=1} \sigma_i f(z_i)] $$ or $$ R_{P,m} = \mathbb E_{s \...
0
votes
1answer
88 views

ELBO interpretation in Variational Autoencoder (VAE) for anomaly detection

How to interpret different ELBO values when checking anomaly detection possibilities of VAE model on different "testing" datasets? The higher the ELBO value of the model when testing it on different ...
0
votes
0answers
34 views

Chernoff bound for multivariate normal distribution

I read in Introduction to Statistical Pattern Recognition about different bounds for Bayes classification errors. It asked to prove that for two multivariate normal distributions, a Chernoff bound ...
0
votes
0answers
11 views

Upper bound on single frequency error based on SSE of the full signal

I have an empirical signal: $y(t) = A_0 + \sum_{i=1}^{M}A_i \cos(\omega_i t + \phi_i) + \epsilon(t)$ The signal is tidal, and so dominated by a small number of frequencies and in particular there is ...
4
votes
1answer
86 views

Fréchet Hoeffding bounds for symmetric random variables

(Edited to clarify the question). The Hakan & Demirtas (2012 doi: 10.1198/tast.2011.10090) approach to approximating Pearson correlation bounds uses the concept of the Fréchet-Hoeffding bounds by ...
2
votes
0answers
38 views

What's the reasoning behind non-seasonal ARIMA model lag-order bounds?

I read some papers on the non-seasonal ARIMA model, and the consensus I've seen is that for ARIMA(p, d, q), p and q should not be greater than 3, maybe 5. What's the reasoning for that? Is it for ...
7
votes
1answer
184 views

Bounds on the difference of correlated random variables

Given two highly correlated random variables $X$ and $Y$, I'd like to bound the probability that the difference $ |X - Y| $ exceeds some amount: $$ P( |X - Y| > K) < \delta $$ Assume for ...
0
votes
0answers
33 views

Is max. Eigenvalue of k-sparse PCA always $\leq$ max. Eigenvalue of normal PCA on same dataset?

Is max. Eigenvalue of k-sparse PCA always less than or equal to the max. Eigenvalue of normal PCA on same dataset? K refers to the number of non zero eigenvalues when the dataset is of dimension n <...
0
votes
0answers
25 views

Expected value of (F(x)/x(1-F(x)) where F(x) is CDF?

I was solving a problem and encountered the following: \sum (f(x)/x) (F(x)/(1-F(x)) Here f(x) is pmf and F(x) is CDF. Is expected value of F(x)/x(1-F(x)) a known function? I found out that f(x)/(1-F(...
0
votes
0answers
125 views

regression with given upper and lower bounds for the target value

I am using several regressors like xgboost, gradient boosting, random forest or decision tree to predict a continuous target value. I have some complementary information like I know my prediction (...
3
votes
0answers
36 views

Concentration inequalities for weighted sums of gaussians

Suppose that $x \sim \cal{N}(0,I_d)$ be a $d$-dimensional standard Gaussian vector and let $x_1,\ldots,x_n$ denote $n$ i.i.d. samples drawn from the same distribution. For some fixed vector $\theta \...
2
votes
2answers
55 views

Understanding the concept of “Bounded in probability”

My statistics book defines the concept of "bounded in probability" in the followng way: ..But doesn't this mean that any sequence of R.V.'s that does not include any R.V.'s with a pdf with infinite ...
4
votes
0answers
43 views

Bounds on probability that a random variable is neither the maximum nor the minimum of a set of random variables

Suppose I have $n$ independent random variables $X_1,\dotsc,X_n$ which are Poisson distributed with $X_i \sim Poi(\lambda_i)$. Without loss of generality, an additional condition $\lambda_1\le\...
1
vote
0answers
25 views

Comparing bound between lasso and OLS

Assume fixed design regression model $Y = \mathbf{X} \beta + \epsilon$ with common assumptions. Lasso estimator $\hat{\beta}_\lambda$ can be shown to have the following bound. For any $\delta > ...
2
votes
0answers
48 views

Bounds on variance and mean of maximum of difference of independent random variables

Suppose $X_1,\dotsc,X_n$ are independent but not necessarily identical random variables. $$Y = \max_{1\le i,j\le n}(X_i-X_j)$$ What upper and lower bounds can be derived for expectation and variance ...
0
votes
0answers
236 views

Upper bound on KL divergence

Is there a maximum (unique?) to the KL divergence between discrete distributions p & q, with the restriction that q is a proper probability distribution? I know KL is unbounded from above when q ...
0
votes
0answers
88 views

Chebychev vs. Hoeffding's inequality in Montecarlo integral evaluation

Suppose that I want to estimate the integral of $ g(x) $ , defined in $[0,1]$ with values in $[0,1]$ via Montecarlo method. There's a smart way to know apriori how large must be N in order to have a ...
3
votes
0answers
38 views

Correlation bounds for uniformly distributed matrix?

For a uniformly or Guassian distributed $M\times N $ matrix. Is there any analytical expression in terms of $M$ and $N$ to estimate the maximum and minimum bounds of correlation between the columns of ...
3
votes
0answers
56 views

Tail bounds for F-distribution (not using $\chi^2$ bounds)

Are there any sharp tail bounds for an $F_{p,q}$ distribution? That is, if $X \sim F_{p,q}$, then for a $t_1,t_2 > 0$, what are the sharpest $\delta_1$ and $\delta_2$ known such that $$P(X > ...
1
vote
0answers
40 views

Bounds on tail conditional expectation of random variable given variance

Given a random variable $X$ with CDF $F(X)$, mean $E(X)=0$, and variance $Var(X) =\sigma^2$, I would like to bound the tail conditional expectation where $X$ is in the tail with probability $1-p$: $E(...
3
votes
1answer
58 views

Minimum of Poissons

Let $X_i\sim\text{Pois}(\lambda_i)$ for $i=1,2,\ldots,n$ and $Y = \min X_i$. Can we show that, for example $\mathbb{E}[Y] \leq f(\lambda,n)\min\lambda_i$ for some $f : (\mathbb{R}^n,\mathbb{N}) \to [0,...
0
votes
0answers
31 views

Is there a method to fit a bound to the plot of an linear inequality?

I have a physical dataset that is bounded by several different processes, and thus the plot takes the form of a linear inequality: I'm specifically interested in studying the upper bound. Is there a ...
3
votes
1answer
90 views

Tightest bounds on sample variance given sample size, mean, minimum, and maximum

For real-valued samples (possibly known to lie in some interval, but without further constraints on them), I am interested in the tightest possible bound on the sample variance $\sigma^2$, given the ...
1
vote
0answers
33 views

How to estimate the bound on a random variable with semi-infinite distribution?

Given samples of a random variable which appears to have a continuous unimodal probability density function that is zero on one side of a bound, what statistical methods are there for estimating that ...
0
votes
0answers
117 views

Relationship between big-Oh and little-Oh

I'm reading Wu, J. (2014), Restoring monotonic power in Wald/LM-type tests. At a certain point in his derivations, an expresion turns out to be $=O_p(\frac{1}{T^{3/10}h^{5/10}}+T^{2/10}h^{20/10})$, ...
0
votes
0answers
41 views

how to use mcmc to generate samples from this posterior density

I tried bivariate random walk gaussian proposal distribution but it converges so slowly. Help!
0
votes
1answer
42 views

Price Analytics - Price range for a product

I have data on weekly, total number of units sold, at what price, any discount, at which store, product details. like below For over 3 years for the Product details across 10 stores. I would like to ...
0
votes
1answer
79 views

About the expectation of a posterior [closed]

I am interested in finding an upper bound for $\mathbb{E}\big( \int_B \frac{f(X| \theta)}{f(X| \theta_0)} \pi(\theta) d \theta\big)$, the $\frac{f(X| \theta)}{f(X| \theta_0)}$ is the likelihood ...
2
votes
0answers
49 views

Constraints on the moments of a bounded probability distribution

Consider a probability distribution with support on $[0,1]$. Suppose the first $n$ raw moments $m_1,...,m_n$ are given. What are the constraints on the $(n+1)^\text{th}$ moment? Obviously we have the ...
6
votes
2answers
418 views

disadvantages variational inference

A lot of methods utilize variational inference for hyperparameter calculation. What are the advantages and disadvantages of variational inference ? (ex: does it guarantee a global optimal ? )
0
votes
1answer
86 views

Need to know pdf of “x/z+sqrt(y^2-x^2)/z” , or any idea about its upper/lower bounds

I need to know the pdf of the following equation or any upper/lower bound would help. Let $X, Y, Z \sim N(0, \sigma^2)$. Then what is the distribution of: $$A=\frac{x+ \sqrt{\mid y^2 - x^2 \mid}}{z}$$...
4
votes
1answer
470 views

Gaussian Multi-Armed Bandits and the UCB Algorithm

I've implemented in MATLAB the UCB algorithm for gaussian bandits with zero mean and unit variance (these means were themselves sampled from a gaussian prior of zero mean and unit variance). Now I ...
1
vote
1answer
159 views

Probability distribution for a limiting value

I have a bunch of measurements $D_1, D_2, ..., D_N$ and their associated uncertainties $\sigma_1, \sigma_2, ..., \sigma_N$. Suppose that the (unknown) true values of the data points are $T_1, T_2, ...,...
1
vote
0answers
52 views

Solve Multivalued function as a Multilevel model?

I have a dependent variable (y) that is bounded between 0 and 2. The ideal value of the variable is 1. Values closer to 0 or 2 are theoretically bad (not outliers). I also have a set of potential ...
2
votes
0answers
66 views

Finding a tighter bound for weighted sum of random variables

Here is the setup of my problem: Let $X_k$ be a sequence of independent random variables in $\mathbb{R}$ with $E(X_n) = 0$ and Var($X_n$)<$\infty$ for all $n$. Let $\{a_k\}$ and $\{A_k\}$ denote ...
3
votes
0answers
113 views

Upper bound for randomly weighted sum of independent random variables

I have a sequence of independent random variables {$\epsilon_j$} with mean 0. I also have another sequence of Bernoulli random variables $\delta_1, \delta_2,\dots$ which are dependent on the previous ...
0
votes
1answer
205 views

log-sum-exp bound to be used in variational inference

I have been reading several papers on using a bound on log sum of exponentials to be used in variational inference. One example of this case that can happen is in correlated topic models. Where to ...
4
votes
1answer
792 views

KL divergence bounds square of L1 norm

In Cover & Thomas, Elements of Information Theory, at the section on Conditional Limit Theorem (11.6), it is proved that the KL divergence bounds the $\cal{L}_1$-norm from above, $\frac{1}{2\ln2}\...
3
votes
1answer
468 views

Lower bound on smallest eigenvalue of covariance matrices

Assume that a class of $p\times p$ covariance matrices is characterized by a parameter $\theta$, i.e, $$\mathbb{F} = \left\{\Sigma(\theta), \theta\in R\right\}$$ Also suppose we know the following ...
4
votes
3answers
158 views

Tails of products of random variables

Let $X$ be a non-negative random variable, and let $Y \sim \chi^2_n / n$ (so that $E(Y) = 1$). $X$ and $Y$ are independent. Note that $X$ and $X\cdot Y$ have the same mean, while $X\cdot Y$ has larger ...
1
vote
1answer
29 views

Bounding cdfs of two distributions with same mean

Let $X$ and $Y$ be two independent non-negative random variables such that $E[X] = E[Y] = \mu$ but $\operatorname{Var}[X] < \operatorname{Var}[Y]$. Can we use only these two moments to show that ...
1
vote
2answers
47 views

lower-bound of data dimension when using a deep learning architecture

I have a (X,Y)=(100,5) dataset (non-image) that I used with a deep linear classifier on Tensorflow to train and evaluate. At the ...
1
vote
0answers
72 views

Under what conditions can I make a bounding argument when I have weak instruments?

The model of interest is $$ y = \alpha + \beta_1 x_1 + \beta_2 x_2 + \epsilon $$ In the population, $E[\epsilon|x_1]\neq 0$ and $E[\epsilon|x_2]\neq 0$, so OLS estimates of $\beta_1$ and $\beta_2$ ...