Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
2 votes
0 answers
23 views

Is there a statistic such that for large sample sizes $a_n (\hat{\theta} - \theta) \sim N(0, \Sigma)$ approximately but $a_n \neq n^{1/2}$?

Various central limit theorems are of the form $a_n(\hat{\theta}-\theta)\sim N(0, \Sigma)$ approximately as $n \to \infty$ and usually $a_n = n^{1/2}$. Are there central limit theorems for statistics ...
user avatar
  • 1,023
0 votes
0 answers
21 views

Given two sequences of random variables whose moments match, does their difference tends to 0?

Suppose you have two sequences of random variables $(X_n)_{n \in \mathbb{N}}$ and $(Y_n)_{n \in \mathbb{N}}$ and you know that for every $n$ $$ \mathbb{E}[X_n^r] = \mathbb{E}[Y_n^r] \quad \forall \, r ...
user avatar
  • 1
0 votes
0 answers
11 views

Multivariate analysis of sample mean and sample variance

$\{X_n\}$ Let be a sequence of iid probability vectors with mean vector$ \mu$ and variance-covariance matrix$ Σ$. In this case, sample variance and sample covariance are defined as follows $S_{n,j}^2=\...
user avatar
0 votes
0 answers
9 views

Establish convergence to normal area

I will try and make my question abstract, since I have two problems of the same overall type. I am given a time series $Y_t$, t=1,...,T. I know that this time series has a (slowly) changing ...
user avatar
  • 21
2 votes
2 answers
37 views

What does $\hat{\theta}_n = \theta + O_p(n^{-1/2})$ mean in terms of rates?

For $\hat{\theta}_n = \theta + O_p(n^{-1/2})$ we have $$\hat{\theta}_n - \theta = O_p(n^{-1/2})$$ Therefore, we have for any $\epsilon > 0$, there exists a finite $M > 0$ and finite $N > 0$ ...
user avatar
  • 321
0 votes
0 answers
25 views

Equivalent definitions of $ L_2$ convergence?

I have been reading up on the convergence of random variables, and I have come across two commonly given definitions of $ L_2 $ convergence: $ \|X_n-X\|_{L_2} \to 0:$ $(1):\left(E|X_n - X|^2 \right)^{...
user avatar
0 votes
0 answers
21 views

Two random sequences converging to one another

Consider two infinite sequences $X_1, X_2, \dots$ and $Y_1, Y_2, \dots$. Suppose that $$\lim_{n\rightarrow \infty} |F_{X_n}(z) - F_{Y_n}(z)| = 0 ~~~~~~~~~~(1)$$ for all $z$ at which both cdf's are ...
user avatar
  • 183
0 votes
1 answer
37 views

Asymptotic normality with weighted sum of objective function $\min_{x} \; f_n(x) + g_n(x)$

Suppose $f_n(x)$, $g_n(x)$ are convex functions w.r.t. $x$ the optimal point of the two problems $\min_x f_n(x)$ and $\min_x g_n(x)$ have asymptotic normality as $n \rightarrow \infty$ they converge ...
user avatar
  • 33
2 votes
0 answers
24 views

Absolute Summability of Coefficients of MA(inf) representation of stationary AR(p)

Not self-study. This is more of a maths question: Let $\Phi(B)X_t = e_t$ be a (weakly) stationary process. Let $$\Phi(B) = \prod_{i=1}^p (1-\alpha_iB)$$ So from stationarity we will have $|\alpha_i|&...
user avatar
  • 2,203
0 votes
0 answers
14 views

why for stochastic gradient one use expectation value?

Why in order to see how far the gradient samples can be far from the true gradient, they use expectation value. For example $$E[\|\nabla f_i(w)-\nabla f(w)\|^2]=E[\|\nabla f_i(w)\|^2]-\|\nabla f(w)\|\...
user avatar
  • 123
0 votes
0 answers
19 views

If the weight and bias gradients are stuck at zero throughout training, is this an indication of dying ReLu?

A high learning rate when combined with a ReLu activation function is known to lead to the 'dying ReLu' problem. Is this a reasonable conclusion to arrive at if the gradient with respect to weights ...
user avatar
0 votes
1 answer
43 views

Convergence in probability exercise

I have to solve the following exercise: Let $X_1,X_2,…,X_n,…$ be a sequence of random variables, with $X_n \sim \text{Uniform}(0,n)$. Also, consider $Y_n= e^{-X_n}$. Does the second sequence ...
user avatar
  • 1
1 vote
0 answers
28 views

Existence of Moments for Linear Regression With Pareto Error

Suppose I have the following model linear regression model: $y = \beta_0 + x_1i\beta_1 + x_2i\beta_2 + e_i$ with $e_i \sim Pareto(k,\alpha)$ Now if $1< \alpha < 2$, I would suppose that the ...
user avatar
0 votes
1 answer
17 views

What is the big $O_p$ of the product between a $O_p(a_n)$ term and a uniformly bounded function?

Suppose $\frac{1}{n}\sum_{i=1}^n \hat{\theta}_i^2 = O_p(a_n)$ and $||f(X)||_{\infty}$ is bounded. What is the big $O_p$ of $\frac{1}{n}\sum_{i=1}^n (\hat{\theta}_i f(X_i))^2$? The way I understand ...
user avatar
  • 321
0 votes
1 answer
32 views

When investigating Monte Carlo convergence, should I reuse previous data?

I am doing Monte Carlo simulations and I want to investigate the convergence. Two versions come to my mind: 1) Doing every trial independently: For each trial, I generate new data independent from any ...
user avatar
  • 203
0 votes
1 answer
52 views

Suppose $\hat{\theta}_1 = O_p(n^{-1/2})$ and $\hat{\theta}_2 = O_p(n^{-1/2})$, what is $\sqrt{\hat{\theta}_1\hat{\theta}_2}$?

Suppose $\hat{\theta}_1 = O_p(n^{-1/2})$ and $\hat{\theta}_2 = O_p(n^{-1/2})$, what is the big $O_p$ for $\sqrt{\hat{\theta}_1\hat{\theta}_2}$? I think $\hat{\theta}_1\hat{\theta}_2 = O_p(n^{-1/2})O_p(...
user avatar
  • 1,613
0 votes
1 answer
57 views

Suppose $E[( \theta-\hat{\theta}_n)^2] = O(n^{-1/2})$. Show that $\frac{1}{n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 = O_p(n^{-1/2}).$

Assume that $E[( \theta-\hat{\theta}_n)^2] = O(n^{-1/2})$. How can I show that $\frac{1}{n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 = O_p(n^{-1/2})?$ What I'm trying to ask is: if the expected value of ...
user avatar
  • 1,613
0 votes
0 answers
17 views

Mixed Model Convergence Issues With LMER

I am fairly new to statistics and I want to run power analyses on my models, so I was hoping to get some insight on model convergence issues. I am using a dataset in which I am examining the effects ...
user avatar
  • 1
0 votes
0 answers
37 views

$(X_i)_{i=1}^n,(Y_i)_{i=1}^m$ , Show if $\frac{n}{m}\to\lambda$, then $\sqrt{n}(\bar X-\bar Y-(\mu_X-\mu_Y))\to_d N(0,\sigma_X^2+\lambda\sigma_{Y}^2)$

Assume that $(X_i)_{i=1,...,n},(Y_i)_{i=1,...,m}$ are independent samples of i.i.d random variables. Assume that $E(X_1)=\mu_X,E(Y_1)=\mu_Y, Var(X_1)=\sigma_{X}^2,Var(Y_1)=\sigma_{Y}^2$. Show if $\...
user avatar
0 votes
1 answer
48 views

A problem in convergence and limit

For any non-negative integer $n$ and some finite $r$, we introduce the notation $n_k$ which indicates the number of $\{X_1, X_2, \cdots , X_n\}$ belonging to the $k$-th distribution type, for $k=1, 2, ...
user avatar
1 vote
1 answer
46 views

A new convergence problem for the conditional expectation

You have risks $X_1$, $X_2$, ... (they are assumed to be independent, but not necessarily identically distributed) and $S_n= X_1 + X_2 + \cdots +X_n$ QUESTION: under what reasonable conditions do we ...
user avatar
0 votes
1 answer
96 views

How to interpret concurvity and wiggliness induced by low or high K knots in GAM models

Two questions: 1. Concurvity for factors =1, is this normal? 2. How do you interpret a partial effect when the effect is linear and lines for confidence intervals are pinched at 0? Is it okay for ...
user avatar
1 vote
0 answers
28 views

Ergodic theorem for Markov chains

I am reading Robert and Casella (2004) on Markov Chain Monte Carlo methods and, in particular, Section 6.7. This contains the ergodic theorem, which is stated as follows, where $S_n(f)$ denotes a ...
user avatar
  • 388
0 votes
0 answers
18 views

Does empirical mean of the product of two random variables converge to the true mean?

If we sample $x_i$'s from $X\sim N(0,1)$, and $y_i$'s from $Y\sim N(0,1)$ (the distributions are set to be normal to make things concrete but they can be any distribution), then is it true that $$\...
user avatar
  • 101
0 votes
0 answers
15 views

What prevents my PyTorch convolutional auto-encoder to converge on some initializations? [duplicate]

I built a small auto-encoder for greyscale images. It is there to make some tests, so I train it often, and I have a strange behavior. On some initialisations, it does not converge. I mean, the MSE ...
user avatar
  • 111
0 votes
0 answers
10 views

Constants in uniform convergence bounds

In the book "Understanding Machine Learning: From Theory to Algorithms" of Shai Shalev-Shwartz and Shai Ben-David on page 73 we have: I'm interested in the value of the constants $C_1$ and $...
user avatar
  • 1,012
0 votes
0 answers
47 views

Sum of squares divided by Variance asymptotic distribution

Consider $X_i$ i.i.d. with mean 0 and variance $\sigma^2$. I am aware that if $X_i$ is normally distributed then $\sum_{i=1}^n X_i^2/\sigma^2 \sim \chi^2_n$. My question is, if $X_i$ has some other ...
user avatar
0 votes
0 answers
10 views

Estimate minimum sample size for a rolling time-serie percentage

I have to calculate the conversion rate for a web page, defined as CVR = orders / clicks This is an e-commerce web page, so clicks arrives to the web page ...
user avatar
2 votes
1 answer
83 views

Stationary distribution of an infinite Markov chain

The exercise is asking for the stationary distribution, the estimated time to get from state $0$ to state $4,$ and to conclude if the chain is time-reversible. So I have the following transition ...
user avatar
  • 21
0 votes
0 answers
23 views

multivariate potential scale reduction factor less than one

I am attempting to implement the multivariate potential scale reduction factor (PSRF) mentioned in this answer and originally described by Brooks and Gelman (1998). When I use a basic Metropolis ...
user avatar
0 votes
1 answer
21 views

For $r>s\geq1$, convergence in $s^{\text{th}}$ mean does not imply convergence in $r^{\text{th}}$ mean

I need a counterexample for the problem: if $r>s\geq1$, convergence in $s^{\text{th}}$ mean does not imply convergence in $r^{\text{th}}$ mean. The definition for convergence in mean is as follows: ...
user avatar
1 vote
0 answers
12 views

Convergence to a distribution when combining random samples

By running an experiment, I get a set of positive real numbers, from which I can draw a histogram or any other density estimate. I can repeat this experiment multiple times; I expect different yet ...
user avatar
  • 183
2 votes
1 answer
180 views

Comparing residuals in two regression models

Let us have two linear models. Let $\alpha_i$ be real numbers same in both. $$ LM1: Y=\alpha_0 + \alpha_1X_1+\alpha_2X_2 + \varepsilon $$ $$ LM2: Y^\prime=\alpha_0 + \alpha_1X^\prime_1+\alpha_2X^\...
user avatar
0 votes
0 answers
19 views

A question about probability convergence (using the Law Of Large Numbers)

I think this problem will help me learn some convergence techniques. Given $(\xi_j)_{j=1}^{\infty} \sim \xi$. Supposse that in probability $$\frac{1}{n}\sum_{j=1}^n \exp ( i s \xi_j ) \to E[\exp ( i ...
user avatar
  • 223
1 vote
2 answers
62 views

Convergence in distribution and convergence in Kolmogorov distance

Let $X, Y$ be two random variables with laws $F$ and $G$ respectively. The Kolmogorov distance between these two laws is defined as: $$ d_{Kol}(F, G) = \sup_{x \in \mathbb R} |\mathbb P(X \leq x) - \...
user avatar
1 vote
0 answers
14 views

$L^2$ convergence of inverse

Let $h$ be some bounded non-negative function. Assume that some random quantity $\mu^N (h)$ be some random quantity with almost sure limit $\mu(h) > 0$. For instance we could have $\mu^N(h) = N^{-1}...
user avatar
  • 131
0 votes
1 answer
28 views

Multilevel (lmer) problems when using weights

I'm having some troubles when running my multilevel model. Specifically, I'm just fitting a starting simple multilevel model with time (2 years, level 1) nested in individuals (level 2). Here a sample ...
user avatar
0 votes
0 answers
11 views

Product of a sequence of RVs (convergence in distribution) and another sequence (convergence in probability)

Suppose that $\{X_n\}\xrightarrow{d} X$ and $\{Y_n\}\xrightarrow{p} 0$. Now assume that $X\sim N(\mu,\sigma^2)$. I think that the following holds $$\{X_nY_n\}\xrightarrow{p} 0$$ My intuitive proof: ...
user avatar
  • 789
0 votes
0 answers
47 views

Bootstrap confidence interval for mean

Take an iid sample $X_1,\dots,X_n$ from a distribution with (unknown) mean $\mu$ and variance $\sigma^2$. Let $\bar{X}_n^b$ be the bootstrap sample mean, and choose $R_n=R_n(X_1,\dots,X_n)$ s.t. $$C_n=...
user avatar
  • 1
0 votes
0 answers
10 views

Accepting Parameter Estimates from Optimization Algorithms that have not Converged

Is there a popular consensus on accepting the solution from an optimization algorithm that has not converged? Suppose you let an optimization algorithm (e.g. gradient descent, Bayesian optimization) ...
user avatar
  • 5,728
1 vote
1 answer
41 views

How should a good cross-validated model look like? [closed]

I am baffled if i should make my model converge at every fold or let it converge at the last fold. Can someone explain me what path should i choose (for a regressional model)? My current "best&...
user avatar
1 vote
0 answers
57 views

Convergence in Quadratic Mean Implies Convergence in Probability Proof Clarification

Wasserman (2004) All of Statistics A Concise Course in Statistical Inference gives the following proof that convergence in quadratic mean ($L_2$ convergence) implies convergence in probability: By ...
user avatar
  • 115
0 votes
0 answers
17 views

Probability of correctly identifying the Bayes class

Consider $X$ being a random variable taking value $\{1, \ldots, K\}$, with probability $p_1 = \frac{1}{K} + \varepsilon$ and $p_k = \frac{1}{K} - \frac{\varepsilon}{K-1}$ for all $k \neq1$. Taking i.i....
user avatar
0 votes
0 answers
16 views

Showing different definitions of almost sure convergence are equivalent

There are a couple different equivalent definitions of almost sure (a.s.) convergence: $\forall \varepsilon>0\quad P(\liminf_{ n\uparrow \infty}\{\|X_n-X\|\leq\varepsilon\})=1$, i.e. $\forall \...
user avatar
2 votes
0 answers
87 views

Large sample properties of classical estimator for single scale parameter

This question was first posted on Math Stackexchange and I was told in the comment it would be a good question on Stats Stackexchange, since it comes from the well-established theory of point ...
user avatar
  • 71
1 vote
0 answers
13 views

Convergence in exponentially weighted total variation implies convergence in total variation?

Am I correct to interpret the following as (also) saying that if a sequence of log-concave densities converges in distribution, then it also converges in total variation? (Meaning that the weighted ...
user avatar
  • 111
0 votes
0 answers
33 views

Convergence of Gaussian random variables

Let $(f_n)$ be a sequence of 0-mean Gaussian densities on $\mathbb{R}^d$ and assume $f$ is limit of $(f_n)$. Question 1 How does one determine the type of convergence by looking at the corresponding ...
user avatar
  • 111
0 votes
1 answer
44 views

Bayesian updating with discrete support

Suppose I have a coin that is either fair ($Pr(H)=0.5$) or biased ($Pr(H)=0.2$). I have a prior probability $Pr(fair)=\tau_{0}$. I observe a sequence of tosses with outcome $X_{1},X_{2},...,X_{n}$, ...
user avatar
0 votes
0 answers
27 views

Bayesian log link regression fails to converge using rjags in r

I am modeling annual densities of birds to estimate population trends in a Bayesian framework, using a log-link regression in rjags. I am modeling long term (2002 - 2020) and short term (2011-2020) ...
user avatar
0 votes
0 answers
14 views

lmer convergence problems in a very simple model

I have a simple psycholinguistic experiment with VAL (word valence) and FREQ (word frequency) as fixed factors and participants (SUBJ) and words (WORD) as random factors: When I tried to estimate a ...
user avatar
  • 1

1
2 3 4 5
21