Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Using Multiple MCMC Chains in Higher Dimensions For Convergence Diagnostics

I have an MCMC problem where I sample from mixture of two multinormal distributions with dimension D. I use Random-Walk Metropolis-Hastings algorithm to sample from that mixture. For the mixture of ...
0
votes
0answers
27 views

Loss function on the training set reaches baseline value and settles

I am training a neural network that needs to solve a regression problem. To give some context, the network needs to predict a target value that lies in the range [0, 0.25] given as input features 4 ...
2
votes
1answer
20 views

Unexpected Zero Variance for an Unbiased Estimator: Is the Estimator Consistent?

$\newcommand{\szdb}[1]{\!\left[#1\right]}\newcommand{\szdp}[1]{\!\left(#1\right)}$ Problem Statement: Let $Y_1, Y_2,\dots,Y_n$ denote a random sample from the probability density function $$f(y)= \...
1
vote
0answers
13 views

Rate of convergence of Machine learning models

I am currently doing some work on the double debiased machine learning algorithm by Chernozhukov et al. 2016. They achieve $ \sqrt{N} $ rate of convergence for estimation of a treatment parameter. ...
3
votes
1answer
35 views

Delta Method around zero is a N(0, 0)

I have this problem: $\sqrt N \hat{\theta} \sim N(0, V)$ where $E(\hat{\theta}) = \theta_{0} = 0$. I must find the asymthotic distribution of $\frac{N}{V}\hat{\theta}^{2}$ but if I use the Delta ...
0
votes
0answers
4 views

What is the relationship between “convergence rate” and “sample efficiency”?

It seems that saying "the algo would converge slowly" and saying "the algo would have a low sample efficiency" mean something similar. Do the two concepts describe different facets ...
1
vote
1answer
37 views

strong law of large numbers for V statistic

Recently, I encountered a problem regarding the a.s.-limit of $\frac{1}{n^2} \sum_{k, \ell = 1}^n ||\boldsymbol X_k - \boldsymbol X_\ell||_2$, where $\boldsymbol X_i$ are $i.i.d.$ sample following ...
1
vote
1answer
37 views

Why does the tail of a Fréchet distribution decay as a power law?

The Fréchet distribution: $$\Phi_\alpha(x)=\begin{cases}0,\; x\leq 0\\e^{-x^{-\alpha}},\; x>0\end{cases}$$ shows a power law decay at the tail (survival): $$1- \Phi_\alpha(x) = 1 -e^{-x^{-\alpha}}\...
0
votes
1answer
33 views

Stop stan when it reaches convergence (Rhat = 1) [closed]

I'm doing a Bayesian analysis, which involves changing the warmup and iterations (many times per day). I wanted to know if there is a loop to automatically change warmups and interactions and stop the ...
5
votes
2answers
220 views

Convergence in $L_1$ counterexample

I am looking for an example of a sequence of r.v. $X_n$ that converges to $X$ in $L_1$, but such that $X_n^2$ does not converge to $X^2$ in $L_1$. Anyone has something in mind?
3
votes
1answer
87 views

Convergence rate of log likelihood ratio

I have come across the following statement in the textbook A course on Large Sample Theory by Ferguson - Chapter 17. Strong Consistency of the Maximum Likelihood Estimates. The likelihood ratio, $L_n(...
1
vote
1answer
65 views

Did the MCMC model converge?

Would you consider model with these MCMC traceplots and R-hat values as converged, and good enough for publication in a peer-reviewed journal? My peers claim convergence is good; the models ran for ...
1
vote
2answers
57 views

Good convergence diagnostic; bad trace plot

I am fitting a multi-level state-space model and am running into a situation where the Gelman-Rubin diagnostic shows acceptable convergence (R-hat < 1.01), but when I look at the trace plots of the ...
0
votes
1answer
30 views

Proof Poisson converges to Normal [closed]

I am looking for a formal proof that, with the CLT transformation, a random variable $Y \sim POI(\lambda)$ converges to a normal distribution ($Z\sim N(0,1)$). I believe this can be formulated as: $$...
0
votes
0answers
14 views

Elo rating system with judges (defined by constant ratings) for massive new player rating assignment

I am thinking of the following variant of Elo rating system (please refer to this question for the background of the conventional Elo rating system), which could potentially expediate the rating ...
0
votes
0answers
11 views

Convergence of Vector argmax

Suppose we have some vector v in $\mathbb{R}^d$. At each timestep t, one of its d elements, chosen randomly, is changed by an external process that moves v towards some unkown v* with high probability ...
0
votes
0answers
17 views

HLM Model failed to converge in R: degenerate Hessian with 1 negative eigenvalues

I have a data that looks like this: Subject Type Score DV 1 type1 -3.1415 10 1 type2 -2.9784 10 2 type1 0.1904 7 2 type2 0.2104 7 ... ... ... ... For each subject, there is two rows, containing ...
1
vote
2answers
42 views

Convergence by probability and transformation

Question 1: If $X_n$ converges in probability to $X$, can I apply the continuous mapping theorem to say that $\dfrac{X_n}{n}$ converges in probability to $\dfrac{X}{n}$? Continuous mapping theorem as ...
5
votes
1answer
64 views

Convergence of uniformely distributed random variables on a sphere

I am reading "Asymptotic Statistics" by A.W van der Vaart and I am stuck with an exercise of chapter 2. Here is the question : for each $n \in \mathbb{N}$, let $U_n$ be uniformly distributed ...
0
votes
0answers
18 views

How to show that the lasso estimator is bounded in probability

The lasso estimator is defined as $$ \hat{\boldsymbol{\beta}}_{n}=\text{argmin}_{\boldsymbol{\beta}}\frac{1}{n}\left\Vert \mathbf{y}-\mathbf{X}\boldsymbol{\beta}\right\Vert_2^2 +\frac{\lambda_{n}}{n}\...
0
votes
0answers
46 views

Markov Autoregression model not converging

I am trying to fit Markov Autoregression model to S&P500 data similar to https://www.statsmodels.org/devel/examples/notebooks/generated/markov_autoregression.html This is the code ...
0
votes
2answers
62 views

Does the convergence of algorithms in machine learning to train set make sense?

Think of the hypothetical situation where we have a y reponse and a scalar input x, we want a function that maps x to y perfectly at least in the training set. This does not make sense, it is just ...
0
votes
1answer
27 views

Bootstrap consistency for maximum likelihood

I'm looking for references (textbook if possible) that treat the strong consistency of bootstrapped maximum likelihood (MLH) or more generally M-estimators. By strong consistency I mean that the ...
8
votes
2answers
201 views

Convergence in distribution to a degenerate distribution

This question came up based on a disagreement I had with a TA. This was the specific example: Let $X_{1},...,X_{n}$ be an iid random sample from a population with pdf $f(x)=3(1-x)^2, 0<x<1$. The ...
1
vote
0answers
13 views

Clustering algorithm which is guaranteed to converge to global minimum [duplicate]

Well known k-means algorithm is not guaranteed to converge to global minimum. It only converges to local minimum. So my question is, what are the clustering algorithms that are guaranteed to converge ...
1
vote
1answer
150 views

Convergence issues and model selection in glmmTMB

Convergence problems in mixed effect models seem to be a common struggle. It is my understanding that they emerge when the likelihood surface is too flat for the optimisation algorithms to find a ...
1
vote
0answers
72 views

ARMA GARCH not converging (rugarch)

I am running an ARMA (1,1) Garch (1,1) model on some log return stock data. I am interested in backtesting this model on every day and using a rolling window of size 300. Whenever I attempt to do this ...
2
votes
1answer
60 views

What is meant by divergence in statistics?

I have learned about the Intuition on the Kullback-Leibler (KL) Divergence as how much a model distribution function differs from the theoretical/true distribution of the data. The two most important ...
0
votes
1answer
33 views

Convergence Distribution and Probability [closed]

Suppose that $|X_n - Y_n|$ converges in probability to 0, and that $X_n$ converges in distribution to X. Show that $Y_n$ converges in distribution to X. Thanks in advance.
1
vote
1answer
33 views

Convergence to 0 in probability for non-iid random variables

Assume $U_k$ are correlated standard normal random variables. Let $R_k := a_k U_k^2$, with $a_k > 0$ and $\sum_{k=1}^{\infty} a_k < \infty$. How can we prove that $S_p:= \frac{1}{p}\sum_{k=1}^{p}...
1
vote
0answers
24 views

Different regularity conditions for finite population CLT

I am having trouble understanding the different regularity conditions for different versions of the finite population central limit theorem. I would greatly appreciate any help or insight anyone has. ...
3
votes
1answer
121 views

Does a Binomial converge to Poisson or Normal?

I have read the answer here. Here the distinction is that If $n\to\infty$ and $p\to0$ while $np$ approaches some positive number $\lambda,$ then the binomial distribution approaches a Poisson ...
0
votes
0answers
40 views

How can I find $ARE_{\hat{\theta}_{n}^{(1)},\hat{\theta}_{n}^{(2)}}$ (in terms of $\tau_1$, $\tau_2$ and $\alpha$)?

Given that $n^{\alpha}[\hat{\theta}_{n}^{(1)}−\theta_0] \xrightarrow{L} \tau_1 H$ and $n^{\alpha}[\hat{\theta}_{n}^{(2)}−\theta_0] \xrightarrow{L} \tau_2 H$, find $ARE_{\hat{\theta}_{n}^{(1)},\hat{\...
9
votes
2answers
511 views

Differences Between the Central Limit Theorem and Consistency

I have recently finished studying the central limit theorem and the idea of consistency. I am still a little fuzzy about them, so I was wondering what are some key similarities and differences of the ...
3
votes
0answers
105 views

As $n \to \infty$, can we 'ignore' a matrix in an expectation that does not depend on $n$?

Let $A_n = \sum_{i=1}^n X_iX_i^T$ and $B = X_1X_1^T$ be random matrices of dimension $m\times m$. Note that the elements of $B$ are dependent on elements of $A$. Note that the vectors $X_i$ are iid ...
1
vote
0answers
26 views

Question about $X$ concentrates to its mean

Suppose for a statistic $X$ of a size-n sample, $E|X-EX|\le f(n)$ for some decreasing function $f(n)$. Can we say $X$ concentrates to its mean at a rate of $f(n)$? I understand concentration rate is ...
1
vote
2answers
36 views

Approximating $E[g(\overline X_n)]$ and want to bound the remainder using some form of CLT or Berry-Essen Theorem

If we have a set $X_1,\dots,X_n$ of iid random variables with finite mean $\mu$ and variance $\sigma$, the CLT says that $\sqrt{n}(\overline X_n - \mu) \stackrel{d}{\to} \mathcal{N}(0,\sigma^2)$. If ...
3
votes
0answers
27 views

Convergence rate about a limit concerning the Poisson CDF

The CDF of a Poisson distribution with rate parameter $\lambda$ is $$ P(n;\lambda)=\sum_{k=0}^n \frac{\lambda^ke^{-\lambda}}{k!}. $$ As $n$ goes to infinity, the CDF would certainly approach 1. Now, ...
2
votes
0answers
34 views

Let $h(t)$ be a continuous function. Show that if $X_n \xrightarrow{D} X$, then $h(X_n) = O_p(1)$

My initial thought on proving this was to use the continuous mapping theorem to conclude $h(X_n) \xrightarrow{D} h(X)$ and then use the fact that convergence in distribution implies tightness in ...
0
votes
1answer
33 views

If $X_n \xrightarrow{D} X$, then $X_n = O_P(1)$

I've seen this result in several places, however, I've yet to find a proof for it and I'm struggling to come up with one on my own. So far I know that I want to show that for all $\epsilon > 0$ ...
1
vote
0answers
21 views

Why is the convergence rate faster for this given approach?

LALR: Theoretical and Experimental validation of Lipschitz Adaptive Learning Rate in Regression and Neural Networks This is a paper that suggests using an adaptive learning rate approach for various ...
1
vote
0answers
25 views

Does $E[\hat \Sigma^{-1}] \to \Sigma^{-1}$ still hold for samples drawn from a non-normal population?

For a sample of observations $\{x_i\}_{i=1}^n$ where $x_i=(x_{i1},\dots,x_{ik})^T$ of a population random vector $X=(X_1,\dots,X_k)^T$, the population covariance is $$ \Sigma = E[(X-E[X])(X-E[X])^T], $...
1
vote
1answer
184 views

Bounding the uniform deviation of the empirical risk from the risk over a finite function class

I am having difficulty interpreting the following theorem from here as a probability statement: Theorem. For all $\delta$ such that $0 < \delta < 1/2$, with proability at least $1 - \delta$ the ...
3
votes
1answer
55 views

Martingale property & limiting distribution for frequency of last names

Suppose that children always inherit their last names from their father (which implies that no new last names are ever created). Pick a last name of interest (e.g. Smith), and let $X_n \in \left[0, 1\...
1
vote
2answers
53 views

Intuition About Gradient Descent Convergence

I know that gradient descent takes steps towards a minimum, but I am having trouble coming up with intuitions about when it will converge. For example, on any given convex function is gradient descent ...
18
votes
2answers
1k views

Optimizing OLS with Newton's Method

Can ordinary least squares regression be solved with Newton's method? If so, how many steps would be required to achieve convergence? I know that Newton's method works on twice differentiable ...
0
votes
0answers
18 views

Convergence of the first moment of empirical distribution

Imagine I have a sequence of random variables $\{x_n\}_{n=1}^{\infty}$. These are not i.i.d. random variables, but an arbitrary sequence. For any $n$, I can define the empirical CDF function $F_n(t) = ...
0
votes
0answers
37 views

Convergence in distribution versus convergence of moments

Suppose we have that a random variable sequence $(X_n)_n$ converges in distribution to a law with mean $\bar{\mu}$ and variance $\bar{\sigma}^2$, or formally $X_n \stackrel{d}{\to} \mathcal{L}(\bar{\...
1
vote
0answers
19 views

Stochastic gradient descent convergence rate

I need to understand the convergence rate notation in the convex optimization context. In every paper that I find, the convergence rate of an algorithm is defined as a function of the number of ...
3
votes
1answer
50 views

How does using PCA speed up supervised learning?

In his popular course, Andrew Ng mentions using PCA to speed up supervised learning (Lecture 14.7). The basic idea is dimensionality reduction, wherein the extremely high-dimensional input features $\{...

1
2 3 4 5
19