Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
1
vote
1answer
17 views

convergence of sample covariance matrix in case sample size depends on dimesion

Let $X_1,X_2,\dots,X_n$ be random sample from $\mathcal{N}_p(\mathbf{0},\mathbf{\Sigma})$ and put $\mathbf{S}=\frac{1}{n}\sum_{i=1}^nX_iX_i^t$, which is sample covariance matrix. If $p<n$, it is ...
0
votes
0answers
29 views

Why isn't my gradient descent code converging to solution for GB2 probability distribution?

I'm running gradient descent code in R on an $n$=10,000 test dataset simulating insurance claims records that follow the Generalized Beta of the 2nd Kind ...
21
votes
6answers
8k views

Intuitive understanding of the difference between consistent and asymptotically unbiased [duplicate]

I am trying to to get an intuitive understanding and feel for the difference and practical difference between the term consistent and asymptotically unbiased. I know their mathematical/statistical ...
2
votes
0answers
18 views

Concentration of top eigenvectors

Let $M$ be a $d \times d$ symmetric matrix with rank $k < d$, write $M = U \Lambda U^T$. Define $\hat{M} = M + Z$, where $z_{ij} \sim N(0, 1 /n )$. Suppose we try to estimate $U$ by taking the top $...
0
votes
1answer
17 views

How to mimic the deterministic convergence via taking the expectation?

Suppose $f: \mathbb{R}^n \rightarrow \mathbb{R}$ is a lower bounded function that satisfies the following property for a given $\{x_k\}_{k\geq0}$ sequence: $$ f(x_k) - f(x_{k+1}) \geq c ||x_{k+1}-{x_k}...
0
votes
0answers
22 views

Proving convergence in distribution between order statistics and quantiles

The random variable of continuous type $X$ has CDF $F(x)$ $X_1, X_2, \cdots, X_n$ is a random sample of size $n$ from the distribution of $X$ Function $h(y)$ is defined as $h(y) = F^{-1}{(1-e^{-y})}I_{...
0
votes
0answers
16 views

Logistic Regression Model does not Converge …How do I interpret the results? [duplicate]

I ran a logistic regression (using R) but for one of my subgroups (by Race), got a warning that the Algorithm does not converge. When I look at my data, for 800 participants, 798 are one outcome, and ...
2
votes
1answer
193 views

Convergence issues and model selection in glmmTMB

Convergence problems in mixed effect models seem to be a common struggle. It is my understanding that they emerge when the likelihood surface is too flat for the optimisation algorithms to find a ...
2
votes
1answer
42 views

Convergence of Empirical Risk Minimizer and True Risk Minimizer

Let $D:= \{ (x_1, y_1), \dots, (x_n, y_n) | x_i \in \mathbb{R}^d, y\in\mathbb{R}\}$ be our dataset. Let $F$ be some function class and $f\in F$. Furthermore, $l$ is some loss function. E.g. the ...
3
votes
1answer
216 views

Asymptotic normality for nonsmooth objective functions

Assume that $f ({\bf x}; \theta): \mathbb{R}^p \times \Theta \to \mathbb{R}$, where ${\bf x}$ is the vector of inputs (with some distribution) and $\theta$ is the vector of parameters. Also, assume ...
0
votes
0answers
18 views

Non-linear curve fitting fail to converge

I'm using the curve fitting toolbox to fit 31 data points to a function: y = a*(1-bexp(-cx)-(1-b)exp(-dx)). (Algorithm: Trust region. The boundaries set for the parameters are: a: 0.25 - Inf, b: 0-1, ...
5
votes
2answers
4k views

A sequence of random variables, how to understand it in the convergence theory?

I am a bit confused when studying the convergence of random variables. All the material I read using $X_i, i=1:n$ to denote a sequence of random variables. Based on the theory, a random variable is a ...
3
votes
1answer
31 views

What is causing the singularity in a glmm with simple random effect?

First time poster but have been very grateful over the past couple months for this forum. First and foremost, I apologize in advance if I am not following the right procedures in asking a question. I ...
1
vote
1answer
798 views

SPSS: GLMM and(adjusted) odds ratio

I am performing a retrospective study and the relative statistic analysis. I am studying the the risk factors for the occurrence of complications during medical procedures. I have 50 subjects ...
3
votes
1answer
166 views

Do fully connected layers in the middle of a network impede optimization?

I submitted a paper that uses an auto-encoder network with several convolutional layers in both the encoder and the decoder and a fully connected layer (FCL) in between. Besides the FCL being useful ...
0
votes
1answer
205 views

Authors frequently mention the convergence of their reinforcement learning algorithms. Do they imply a local or a global convergence?

I frequently come across authors in reinforcement learning papers mentioning that some or the other algorithm converges. Do they mean a local convergence or a global convergence? What do they ...
0
votes
0answers
16 views

Using Multiple MCMC Chains in Higher Dimensions For Convergence Diagnostics

I have an MCMC problem where I sample from mixture of two multinormal distributions with dimension D. I use Random-Walk Metropolis-Hastings algorithm to sample from that mixture. For the mixture of ...
2
votes
1answer
519 views

Convergence in probability: Does squeeze theorem apply?

Does squeeze theorem apply in convergence in probability? My statistics reference (where it talks about convergence in probability and its condition) does not cite it (but does seem to apply it), but ...
0
votes
0answers
29 views

Loss function on the training set reaches baseline value and settles

I am training a neural network that needs to solve a regression problem. To give some context, the network needs to predict a target value that lies in the range [0, 0.25] given as input features 4 ...
2
votes
1answer
21 views

Unexpected Zero Variance for an Unbiased Estimator: Is the Estimator Consistent?

$\newcommand{\szdb}[1]{\!\left[#1\right]}\newcommand{\szdp}[1]{\!\left(#1\right)}$ Problem Statement: Let $Y_1, Y_2,\dots,Y_n$ denote a random sample from the probability density function $$f(y)= \...
1
vote
2answers
42 views

Convergence by probability and transformation

Question 1: If $X_n$ converges in probability to $X$, can I apply the continuous mapping theorem to say that $\dfrac{X_n}{n}$ converges in probability to $\dfrac{X}{n}$? Continuous mapping theorem as ...
1
vote
0answers
13 views

Rate of convergence of Machine learning models

I am currently doing some work on the double debiased machine learning algorithm by Chernozhukov et al. 2016. They achieve $ \sqrt{N} $ rate of convergence for estimation of a treatment parameter. ...
3
votes
1answer
38 views

Delta Method around zero is a N(0, 0)

I have this problem: $\sqrt N \hat{\theta} \sim N(0, V)$ where $E(\hat{\theta}) = \theta_{0} = 0$. I must find the asymthotic distribution of $\frac{N}{V}\hat{\theta}^{2}$ but if I use the Delta ...
0
votes
0answers
6 views

What is the relationship between “convergence rate” and “sample efficiency”?

It seems that saying "the algo would converge slowly" and saying "the algo would have a low sample efficiency" mean something similar. Do the two concepts describe different facets ...
1
vote
1answer
42 views

strong law of large numbers for V statistic

Recently, I encountered a problem regarding the a.s.-limit of $\frac{1}{n^2} \sum_{k, \ell = 1}^n ||\boldsymbol X_k - \boldsymbol X_\ell||_2$, where $\boldsymbol X_i$ are $i.i.d.$ sample following ...
1
vote
1answer
38 views

Why does the tail of a Fréchet distribution decay as a power law?

The Fréchet distribution: $$\Phi_\alpha(x)=\begin{cases}0,\; x\leq 0\\e^{-x^{-\alpha}},\; x>0\end{cases}$$ shows a power law decay at the tail (survival): $$1- \Phi_\alpha(x) = 1 -e^{-x^{-\alpha}}\...
21
votes
6answers
4k views

Does the normal distribution converge to a uniform distribution when the standard deviation grows to infinity?

Does the normal distribution converge to a certain distribution if the standard deviation grows without bounds? it appears to me that the pdf starts looking like a uniform distribution with bounds ...
6
votes
1answer
642 views

Does the definition of regular estimator depend on the rate of convergence? If not, should it?

The definition of regular estimator in my lecture notes is: Let $X_1^{(n)}, \dots, X_n^{(n)} \overset{iid}{\sim} P_n \sim \mathcal{P}(\Theta)$ where $\mathcal{P}(\Theta)$ is a regular parametric ...
3
votes
1answer
93 views

Convergence rate of log likelihood ratio

I have come across the following statement in the textbook A course on Large Sample Theory by Ferguson - Chapter 17. Strong Consistency of the Maximum Likelihood Estimates. The likelihood ratio, $L_n(...
5
votes
2answers
229 views

Convergence in $L_1$ counterexample

I am looking for an example of a sequence of r.v. $X_n$ that converges to $X$ in $L_1$, but such that $X_n^2$ does not converge to $X^2$ in $L_1$. Anyone has something in mind?
5
votes
1answer
121 views

Convergence of multivariate ECDF

Gilvenko-Cantelli assures uniform a.s. convergence of univariate ECDF. My questions are: Are there similar assurances for multivariate ECDF? How is the rate of convergence dependent on the ...
0
votes
1answer
36 views

Stop stan when it reaches convergence (Rhat = 1) [closed]

I'm doing a Bayesian analysis, which involves changing the warmup and iterations (many times per day). I wanted to know if there is a loop to automatically change warmups and interactions and stop the ...
1
vote
2answers
169 views

Prove convergence in distribution, probability, or quadratic mean for a sequence of binary variables that depend on another binary variable

Suppose that $X$ has the support set $\{1, -1\}$, and $P(X = 1) = P(X = -1) = 0.5$. Suppose that $X_n$ has the support set $\{X, e^n\}$, and $P(X_n = X) = 1 - \frac{1}{n}$ $P(X_n = e^n) = \frac{1}...
1
vote
1answer
67 views

Did the MCMC model converge?

Would you consider model with these MCMC traceplots and R-hat values as converged, and good enough for publication in a peer-reviewed journal? My peers claim convergence is good; the models ran for ...
1
vote
2answers
67 views

Good convergence diagnostic; bad trace plot

I am fitting a multi-level state-space model and am running into a situation where the Gelman-Rubin diagnostic shows acceptable convergence (R-hat < 1.01), but when I look at the trace plots of the ...
3
votes
1answer
431 views

Saddle-free Newton method for SGD - while Newton attracts saddles, is it worth to actively repel them?

While 2nd order methods have many advantages, e.g. natural gradient (e.g. in L-BFGS) attracts to close zero gradient point, which is usually saddle. Other try to pretend that our very non-convex ...
37
votes
6answers
12k views

Intuitive explanation of convergence in distribution and convergence in probability

What is the intuitive difference between a random variable converging in probability versus a random variable converging in distribution? I've read numerous definitions and mathematical equations, ...
0
votes
1answer
31 views

Proof Poisson converges to Normal [closed]

I am looking for a formal proof that, with the CLT transformation, a random variable $Y \sim POI(\lambda)$ converges to a normal distribution ($Z\sim N(0,1)$). I believe this can be formulated as: $$...
0
votes
0answers
14 views

Elo rating system with judges (defined by constant ratings) for massive new player rating assignment

I am thinking of the following variant of Elo rating system (please refer to this question for the background of the conventional Elo rating system), which could potentially expediate the rating ...
0
votes
0answers
12 views

Convergence of Vector argmax

Suppose we have some vector v in $\mathbb{R}^d$. At each timestep t, one of its d elements, chosen randomly, is changed by an external process that moves v towards some unkown v* with high probability ...
8
votes
2answers
210 views

Convergence in distribution to a degenerate distribution

This question came up based on a disagreement I had with a TA. This was the specific example: Let $X_{1},...,X_{n}$ be an iid random sample from a population with pdf $f(x)=3(1-x)^2, 0<x<1$. The ...
9
votes
1answer
430 views

MLE, regularity conditions, finite and infinite parameter spaces

The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below. The set up is as follows: we are consider a parameters space ...
19
votes
2answers
4k views

Does log likelihood in GLM have guaranteed convergence to global maxima?

My questions are: Are generalized linear models (GLMs) guaranteed to converge to a global maximum? If so, why? Furthermore, what constraints are there on the link function to insure convexity? My ...
5
votes
1answer
64 views

Convergence of uniformely distributed random variables on a sphere

I am reading "Asymptotic Statistics" by A.W van der Vaart and I am stuck with an exercise of chapter 2. Here is the question : for each $n \in \mathbb{N}$, let $U_n$ be uniformly distributed ...
0
votes
0answers
20 views

HLM Model failed to converge in R: degenerate Hessian with 1 negative eigenvalues

I have a data that looks like this: Subject Type Score DV 1 type1 -3.1415 10 1 type2 -2.9784 10 2 type1 0.1904 7 2 type2 0.2104 7 ... ... ... ... For each subject, there is two rows, containing ...
26
votes
3answers
12k views

Extreme Value Theory - Show: Normal to Gumbel

The Maximum of $X_1,\dots,X_n. \sim$ i.i.d. Standardnormals converges to the Standard Gumbel Distribution according to Extreme Value Theory. How can we show that? We have $$P(\max X_i \leq x) = P(...
0
votes
0answers
18 views

How to show that the lasso estimator is bounded in probability

The lasso estimator is defined as $$ \hat{\boldsymbol{\beta}}_{n}=\text{argmin}_{\boldsymbol{\beta}}\frac{1}{n}\left\Vert \mathbf{y}-\mathbf{X}\boldsymbol{\beta}\right\Vert_2^2 +\frac{\lambda_{n}}{n}\...
0
votes
2answers
63 views

Does the convergence of algorithms in machine learning to train set make sense?

Think of the hypothetical situation where we have a y reponse and a scalar input x, we want a function that maps x to y perfectly at least in the training set. This does not make sense, it is just ...
0
votes
0answers
57 views

Markov Autoregression model not converging

I am trying to fit Markov Autoregression model to S&P500 data similar to https://www.statsmodels.org/devel/examples/notebooks/generated/markov_autoregression.html This is the code ...
0
votes
1answer
28 views

Bootstrap consistency for maximum likelihood

I'm looking for references (textbook if possible) that treat the strong consistency of bootstrapped maximum likelihood (MLH) or more generally M-estimators. By strong consistency I mean that the ...

1
2 3 4 5
19