Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Neural network training converges for several epochs, then diverges badly

I have a VGG-like network that I have trained from scratch on a multi-class dataset of my own. The results suggested there were probably some data errors somewhere, so I thought I would train the same ...
0
votes
0answers
15 views

What's the relationship between these two statements?

Suppose I have a set $A\subset R^2$ and an estimator for set $A$ denoted as $\widehat{A}$. Let $\widehat{A}\Delta A$ denote the symmetric difference between set $\widehat{A}$ and set $A$. For some ...
0
votes
1answer
36 views

How to show the rate of convergence of this maximum likelihood estimator is $n^{-1}$?

Suppose I have data $\{X_i\}_{i=1}^{n}\sim \text{Uniform}[0,\theta_0]$. We know that the maximum likelihood estimator of $\theta_0$ is $\widehat{\theta}=\max\{X_1,...,X_n\}$. How to show that $\...
0
votes
0answers
4 views

What are the most likely reasons for convergence failure in back-fitting algorithm (GAM)?

I'm trying to build a binomial model using proc gam in SAS. When I include certain predictors as splines in the model , I get an error that the scoring algorithm ...
2
votes
0answers
17 views

Show that a sequence of random variables diverges to infinity in probability

I have sequences of real-valued random variables $\{X_T\}, \{Y_T\}$ and a sequence of real numbers $\{a_T\}$. As $T\rightarrow\infty$, I know that $$ a_T \rightarrow \infty $$ and $$ X_T \overset{d}{\...
1
vote
0answers
21 views

What does weak convergence mean for a stochastic process?

I am reading a paper in which stochastical processes $\{\mathcal{H}_T(u)\}_{u\in[0,1]}$ and $\{\mathcal{H}(u)\}_{u\in[0,1]} $ on [0,1] with $u$ as a time-index occur. There is a theorem which states ...
1
vote
0answers
34 views

Central limit theorem implications about $\bar{X}$

The central limit theorem (CLT) and law of large numbers (LLN) look to make the same claim. $$\text{CLT: }\sqrt{n}\big(\bar{X}_n-\mu\big) \rightarrow N(0,\sigma^2) $$ $$\text{LLN: }\bar{X}_n \...
1
vote
0answers
84 views

Convergence in Probability (Analytical Solution Verification)

Problem: Let $X_1,X_2,\cdots$ be independent random variables that are uniformly distributed over $[-1,1]$. Show that the sequence $Y_1,Y_2,\cdots$ converges in probability to some limit, and identify ...
1
vote
1answer
40 views

Why is this true?

suppose $T$ is a binary variable and $x$ is a continuous scalar, and $g(x)=E[T|x]$ is the conditional expectation of $T$. Suppose I estimate $g(x)$ using kernel regression $\widehat{g}(x)=\frac{\sum_{...
1
vote
1answer
35 views

Convergence example

I'm studying convergence in my probability class and I'm asked to show if there exists any convergence for the following sequence of random variables: $$\left\{\frac{W_n}{ln(n)}\right\}_{n\geq1} \ s.t....
0
votes
2answers
83 views

Convergence of a semiparametric estimator - a doubt

Suppose we have a parametric continuous function of $x\in\mathbb{R}$ with d-dimensional parameter $\theta$ $$g(x;\theta)$$ we also have have an n-dimensional sample if i.i.d. observations of X. With ...
36
votes
6answers
3k views

Debunking wrong CLT statement

The central limit theorem (CLT) gives some nice properties about converging to a normal distribution. Prior to studying statistics formally, I was under the extremely wrong impression that the CLT ...
1
vote
1answer
41 views

A linear process $x_{t}$ satisfies $\sum\limits_{j \in \mathbb Z}\lvert \gamma(j) \rvert < \infty$

A linear process $x_{t}$ is the weighted sum of white noise variates $(w_{t})_{t}$, i.e. $$x_{t}=\mu+\sum\limits_{k \in \mathbb Z}\psi_{k}w_{t-k}$$ such that $$ \sum\limits_{j \in \mathbb Z}\lvert \...
3
votes
2answers
59 views

Convergence in distribution of sum of random variables

Let $\{x_{1,n}\}_{n\in\mathbb{N}},...,\{x_{k,n}\}_{n\in\mathbb{N}}$ be random sequences of zero mean random variables satisfying $$x_{1,n}\overset{d}{\to} N(0,\sigma^2_1),\cdots, x_{k,n}\overset{d}{\...
0
votes
1answer
32 views

Why does my Gibbs sampler find two optimals?

[EDITED] I am using a Gibbs Sampler to find a Bayesian optimization to my multilevel (hierarchical) model (2 levels). However, when I run multiple chains (each chain having different starting values) ...
1
vote
1answer
38 views

multilevel modeling with lmer(): understanding failure to converge in a toy example

I am trying to get a deeper understanding of failures to converge in multilevel models that I estimate with lmer(). "Failure to converge" is vague; I want to be ...
0
votes
0answers
17 views

Is this proof of convergence in probability to zero correct?

I want to show that $A=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\widehat{B}_{i}-B_{i})X_i$ converges in probability to 0, where $B_i=E(C_i|Z_i)$ and $C_i$ is i.i.d. binary and $Z_i$ is a discrete random ...
0
votes
1answer
17 views

Loss is stuck at 67% and wont converge even with large epoch and early stopping criterion [duplicate]

I am training a very simple 2D dataset with 2 features. Its tabular data and contains only numeric information. I tried using keras to train a neural network but the performance does not bulge. I ...
1
vote
2answers
90 views

Central limit theorem seems counterintuitive given Law of large number

From what I understand, the Central limit theorem says the sample mean is distributed normally when sample number tends to infinity. However, the Law of large number says sample mean converges in ...
2
votes
0answers
34 views

Asking for feedback on the application of a Central Limit Theorem

Let $\{X_{n,i}:1\leq i \leq d_n\}$ be a triangular array of mean zero random variables where $d_n$ is a positive increasing sequence ($d_n\leq n$). Under some conditions, a Central Limit Theorem ...
2
votes
2answers
33 views

For arbitrary random variable $Z$, prove $P(\lvert Z1_{B^{c}}\lvert > \epsilon) \leq P(B^{c})$?

This question is asked to understand proof of Lemma 9.15 from Keener. For arbitrary random variable $Z$, show that $$P(\lvert Z1_{B^{c}} \lvert > \epsilon) \leq P(B^{c})$$ for event $B$ and ...
0
votes
0answers
9 views

Deciding whether the autocorrelation plot shows a good sign of convergence?

I am wondering whether the autocorrelation plot (from MCMC sampling) shows a good sign of convergence when there is some autocorrelation until the 4~5th lags (at 1st lag 0.6, at 2nd lag 0.26, at 3rd ...
1
vote
1answer
51 views

convergence in distribution of sum of two normals

Let $x_n, y_n$ be sequences of zero mean random variables, not necessarily i.i.d. Suppose that there are finite $\sigma_1^2,\sigma_2^2$ such that $$x_n\overset{d}{\to} N(0,\sigma_1^2), $$ and $$y_n\...
2
votes
2answers
45 views

Question on Convergence in distribution

Belyaev and Sjöstedt-de Luna introduced the notion of weakly approaching sequences of distributions, generalizing the weak convergence without imposing the limiting distribution. Definition. Two ...
2
votes
1answer
30 views

convergence of an average of consistent estimators?

Let $\frac{1}{n}\sum_{i=1}^n X_i^j \overset{\text{p}}{\to} \mu^j$ for each $j$ (as $n \to \infty$ ). Under what conditions can we guarantee that $$ \frac{1}{nm}\sum_{j=1}^m\sum_{i=1}^nX_i^j \overset{\...
1
vote
1answer
18 views

Is the independence of this sequence of random variables not implicitly given when we define their probability distributions?

In this post, the user asks whether the following random variable converges to $0$ almost surely: $X_n = \begin{cases} 0, & \text{with probability 1 - $\frac{1}{2n}$,} \\ n, & \text{with ...
1
vote
0answers
12 views

How many days should I measure a variable to get a good estimate of 'average' behavior?

I have data for individuals, spread over a number of days. My dataset is very large: it covers way more days than what is typically feasible to collect. I want to use this to create a 'benchmark' of ...
1
vote
1answer
191 views

Convergence of random sample vs. Latin Hypercube Sampling

Latin Hypercube Sampling, by concept, should be able to yield convergence of an estimate of output at a lower number of samples than random sampling. With the model I am working on, I'm continuously ...
1
vote
1answer
31 views

Variance of Sample Mean for a Positively Correlated Sample

Suppose we have a sample of n observations which are positively correlated with correlation matrix given by $\sigma^2$ as diagonal entries and $a\sigma^2$ as off-diagonal entries. Then we should get ...
1
vote
0answers
21 views

How are the convergence conditions/KKT conditions for the soft-margin SVM derived

With reference to CS229 lecture notes here, I do not understand these equations, which apparently signify the convergence conditions/KKT conditions for the SMO algorithm: I understand that the ...
0
votes
0answers
10 views

Does almost sure convergence imply uniform convergence

Does a sequence of random variables that converges almost surely also converge uniformly? I already know that the converse is true but I feel that this is false.
1
vote
1answer
44 views

What does it mean when glm algorithm doesn't converge but still gives results

I'm running many glm models in R (negative binomial regression to be specific) to a fairly large dataset (N = 175,000) with the intention of performing a specification curve analysis. For my case, ...
0
votes
0answers
19 views

What pitfalls should we avoid with Heidelberger-Welch convergence

I'm working through validating a Bayesian mixture model for multi-species occupancy with a collaborator. Initially, we relied on coda::heidel.diag to alert us to ...
2
votes
1answer
22 views

How to show that $X_n/n$ approaches a constant as $n \to \infty$ if $X_n \tilde\ \chi_{n-p}^2$

Page 18 here states that if $X_n \sim \chi_{n-p}^2$ with fixed $p$, then $X_n/n$ approaches a constant. How do I show this?
4
votes
2answers
95 views

Why is the convergence rate important?

Basically I am trying to find the intuition behind why in some theorems we care so much about the convergence rate. For example, many theorems state that the convergence rate is $\sqrt{n}$ Why ...
0
votes
0answers
8 views

Understanding proof of k-nearest neighbors convergence to Bayes Decision boundary

I'm working on the proof that under sufficient regularity conditions k-nearest neighbor converges to the Bayes Decision boundary as n, the number of data points increases. I have read that 1-nearest ...
0
votes
0answers
17 views

MCMC convergence of phylogenetic random effect chains

I have run 100 zero-altered Poisson models, using the birds of the world as my data points and 100 different phylogenetic trees to account for phylogenic correlation. I checked model convergence using ...
1
vote
0answers
15 views

Adam converges while SGD does not improve at all

I am trying to build a model based movie recommendation system with a neural network. The architecture looks as follows: ...
0
votes
0answers
28 views

What is the difference between a non-central limit theorem and the usual central limit theorems?

I'm reading a paper where the authors prove the following theorem. They then say that this constitutes a non-central limit theorem for the variables in question. Since I have never heard this term (...
1
vote
1answer
31 views

Convergence of two sequences in probability implies joint convergence in probability- problem with proof

Let $X_n\xrightarrow{P} X$ and $Y_n\xrightarrow{P}Y$, then we have $(X_n,Y_n)\xrightarrow{P}(X,Y)$. In process of proof we have the following: $$ \mathbb{P}(\{||(X_n,Y_n)-(X,Y)||\geqslant\epsilon\})\...
-1
votes
1answer
49 views

What makes MCMC converge?

Here is what I have learned about MCMC recently 1) We first propose a likelihood function that describes our problem (Binomial) 2) We define a conjugate prior (Beta) and posterior distribution (Beta-...
3
votes
1answer
53 views

How can these two expressions both converge in distribution to N(0, 1)?

In All of Statistics, chapter 11 (pg. 183), Larry Wasserman states in his description of the Wald Test: We are testing the null hypothesis $ \hat{\theta} = \theta_0 $ versus the alternative ...
3
votes
1answer
94 views

Probability distributions associated to the logarithm numeration system

The most elementary logarithmic numeration system is defined as follow. Any random number $X \in [0, 1]$ can be represented uniquely as $$X=\log_3(A_1 + \log_3(A_2+\log_3 (A_3 + \cdots)))$$ with $A_k \...
0
votes
0answers
20 views

Convergence in distribution implies convergence in probability in proof of Delta Method

I'd like verification that my proof of the below claim is correct. This is part (a) of exercise 5.4.3 of Casella and Berger. I found a similar question on this forum but the response used a different ...
1
vote
0answers
13 views

Are there methods or considerations for real-time Gelman-Rubin diagnosis while running multiple mcmc chains in parallel?

I have a Metropolis-Hastings algorithm in Python and I parallelized it with the multiprocessing library. However, at this moment I can only do Gelman-Rubin diagnosis when all generated parallel chains ...
0
votes
0answers
25 views

Does convergence in probability imply $\sqrt{n}$-consistency?

Consider a linear model $y=X\beta+\varepsilon$, and take $\tilde\beta$ as an estimator of the population parameter $\beta$. If $\left\|\tilde\beta-\beta\right\|_2\rightarrow0$ in probability, does ...
9
votes
1answer
126 views

Having a hard time with the law of the iterated logarithm

Let's say you have infinitely many i.i.d. Bernouilli variables $X_1, X_2, \cdots$ of parameter $p=\frac{1}{2}$. For instance, the binary digits of a random number. Let $S_n = X_1 + \cdots X_n$. The ...
0
votes
0answers
17 views

Why am I getting a good accuracy even with an overfitted CNN model?

I got a bad fit for my model and no convergence happened during the time of training. However, I still got an accuracy of 97%. Then, I trained my other model and got an accuracy of 91%. However the ...
1
vote
0answers
31 views

Almost sure convergence of the minimum eigenvalue of a sample covariance matrix

I was wondering if someone could provide a reference to the following result. Consider the $p\times p $ sample matrix $$\frac{1}{n} \sum_{i=1}^n x_i x_i',$$ where $x_i$ are i.i.d. $p\times 1$ random ...
1
vote
0answers
17 views

Glmer mixed model with many parameters. Better convergence or more precise estimation?

I have a complex model logistic model of the form: cbind(Cases, Pop - Cases) ~ X * log(X) + (X * log(X) | Country) The model has difficulties converging given ...

1
2 3 4 5
17