# Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

800 questions
Filter by
Sorted by
Tagged with
23 views

### Why does network training loss converge to data variance in overfit?

I'm trying to create some sanity checks for a big project i have, so for this purpose I have trained a toy example of a fully connected NN with some vector of scalars as input, and a regression task (...
17 views

### Rates of convergence for estimating population mean squared error

Suppose I have an i.i.d. sample $\{(Y_i, X_i)\}_{i=1}^n$ on which I am trying to estimate a conditional expectation model: $$Y = g(X) + \varepsilon,\quad \mathbb E[\varepsilon | X] = 0$$ There is a ...
29 views

### Does large Rhat for one parameter mean that marginal posterior for another cannot be trusted?

I'm using Stan to fit a model on some simulated data. The model has several parameters and one of them, say $\alpha$, has a large Gelman-Rubin statistic value, $\hat{R} > 1.1$. This is however a ...
17 views

### Basic PageRank algorithm

I am trying to understand the pageRank algorithm by reading the original article: http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf I have some issues understanding the algorithm at paragraph 2.6 . ...
45 views

145 views

### Is a singular fit with no correlations near +/- 1 or variances of zero, a false positive?

I sometimes get a "singular fit" warning when fitting mixed models, yet when I inspect the variance-covariance matrix of random effects, there are no correlations near -1 or +1, nor any standard ...
104 views

### Convergence time for LSTM and Vanilla feed-forward NN training/validation errors

While learning myself, I am doing a simple example of traffic forecasting with LSTM, comparing with vanilla feedforward NN (FFNN). I observed the following When I have a large number of training ...
151 views

### How to prove convergence in probablity

Let $Y_1$, $Y_2$, ... be a sequence of random variables such that $P(Y_n=\frac{1}{n})=1-\frac{1}{n^2}$ and $P(Y_n=n)=\frac{1}{n^2}$. Does $Y_n$ converge in probability? I am stuck because I don't ...
113 views

172 views

### Does $X\stackrel{d}\to X_1$ and $Y\stackrel{d}\to Y_1$ imply $X+Y\stackrel{d}\to X_1+Y_1$?

Let $X,X_1, Y, Y_1$ be random variables. If $X\to X_1$ and $Y\to Y_1$ converge in distribution, does $X+Y\to X_1+Y_1$ in distribution?
17 views

### Convergence of Diffusion Process Monte-Carlo

Let $X_t$ be a $d$-dimensional diffusion process initialized at $x \in \mathbb{R}^d$; given as the strong solution to the SDE $$X_t = x + \int_0^t a(t,X_t)dt + \int_0^t b(t,X_t)dW_t;$$ where $a$ and ...
308 views

### Is there a stronger Universal Approximation Theorem for LSTMs?

The Universal Approximation Theorem says that under certain conditions on your activation function, you can approximate any bounded continuous function with a feedforward neural network. I believe ...
125 views

### Convergence in distribution of a sequence indexed by a random variable

Let $(X_n(\theta))_{n \geq 1}$ be a sequence of random variables with value in $\mathbb{R}^q$ indexed by a parameter $\theta \in \Theta \subset \mathbb{R}^q$. Suppose that for all $\theta \in \Theta$: ...
46 views

### Can we use Gelman-Rubin diagnostic to assess convergence of parallel tempered chains in MCMC?

I know that the principle behind the Gelman-Rubin diagnostic is comparing within-chain and between-chain variances and if the potential scale reduction factor is less than, say 1.1 or 1.05 then the ...
20 views

### Model loss stays the same for hours before dropping

I'm training a CNN to colorize images. The model I have is not incredibly deep, and should work fine on the card I'm training on (2080 TI). Initially, I suspected the model was flawed in some way ...
I am reading Larry Wasserman's All of Statistics and exercise 2 in chapter 6 asks for a proof that given sequence of random variables $X_1, X_2, \dots$, show that $X \xrightarrow{\text{QM}} b$ if ...