Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
1
vote
0answers
9 views

Bootstrap: mixing independent and time-series data together

I have a very computationally heavy simulator (large-scale agent-based transport simulation), which usually takes up to 5 days of run time in a large computer. The results are probabilistic, so ...
7
votes
2answers
1k views

Why is the limit of a Chi squared distribution a normal distribution?

My professor claimed that $\lim_{p\to\infty}\chi^2_p$ has a normal distribution. The claim was made on the basis of the Central Limit Theorem: as $p\to\infty$, we have a Normal$(p\mu, p^2\sigma^2)$. I ...
1
vote
2answers
136 views

Show that the distribution of $\frac{1}{\sqrt{n}}\sum_{i=1}^n(X_i^2-3)$ is normal

Let $X_1,\ldots,X_n$ be i.i.d. variables with $\mathbb{E}[X_i]=0$ and $\mathbb{V}[X_i]=3$ and assume that $\mathbb{E}[X^4_i]<\infty$, show that $$ \frac{1}{\sqrt{n}}\sum_{i=1}^n(X_i^2-3) $$ ...
0
votes
0answers
28 views

Which is the better estimator for standard deviation?

Let $X_i \sim^{\textrm{iid}} N(\mu, \sigma^2)$. If I have measured $n$ values of $\textrm{std}(X_i)$ as $\sigma_1,\cdots,\sigma_n$, then what is the better estimator for $\sigma$: $$\hat{\sigma}_1 = ...
0
votes
1answer
35 views

Probability of an event with probability 0 happening at least once in infinite trials

This question here is confusing me a lot. To summarize, let's say you have $\text{i.i.d. }X_i \sim U(0, 1), i = 1,2,\ldots, n.$ The question shows that $Y_i = \max(X_1, \ldots, X_i) \rightarrow 1$ in ...
0
votes
1answer
33 views

How can we conclude that an optimization algorithm is better than another one for a problem at hand

When we test a new optimization algorithm for a particular problem at hand, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,...
2
votes
0answers
124 views

Proving Linear Regression with Gradient Descent Converge to OLS estimates

Problem I am having trouble showing that the parameters $\theta\in \mathbf{R}^{m}$ for Linear Regression converge to the classic OLS estimates using gradient descent. Please find below my attempt: ...
0
votes
0answers
56 views

convergence in distribution when parameters converges almost surely

let $X_n$ be sequence of random variables with the associated distribution $N(\mu_n,\beta_n+E)$. That is a sequence of normally distributed random variables with changing mean and variance. $\beta_n$ ...
0
votes
1answer
268 views

How many iterations are too many?

I have the following model: ...
0
votes
0answers
23 views

Issue with proof in Statistical Theory related to Beta and Binomial distributions [duplicate]

Assume $X_n$ is distributed $\text{Beta}(1/n, 1/n)$ and $X$ is distributed as $\text{Binom}(1,1/2)$. Show that $X_n$ converges to $X$ in distribution. I'm having some issue with this question. I ...
1
vote
0answers
29 views

convergence and efficiency of mcmc chains and estimation of covariance matrix

I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
1
vote
0answers
449 views

Time complexity of batch gradient descent

I am read http://papers.nips.cc/paper/4937-accelerating-stochastic-gradient-descent-using-predictive-variance-reduction.pdf paper. It states that "Due to the poor condition number, the standard batch ...
0
votes
1answer
77 views

Real Analysis like convergence of Loss Functions [closed]

Here's one thing i noticed. In Elementary Real Analysis, when we say that a sequence $s_n$ converges to a point $s$, we first set an $\epsilon > 0$ such that for large $N \in \mathbb{N} $ $| s_n - ...
0
votes
0answers
23 views

Uniform Convergence of Partial Autocorrelation

I've been facing the problem of estimating a large PACF as the sample size grows. My question is whether we can guarantee that the partial autocorrelation, estimated by the projection $X_t = \sum_{...
0
votes
0answers
98 views

Convergence of Modified Expectation-Maximisation Algorithm - interpreting language of question

We're going to consider a modified E-M algorithm and its convergence properties. To do so, we will first need to review the convergence of the standard E-M algorithm as I'll need to refer back to it. ...
6
votes
2answers
187 views

Tail bound of beta distribution when $\alpha$ is sufficiently close to zero while $\beta$ greater than 1

I am interested in finding sharp enough bound for tail probability $\Pr(X\gt t)$ given $X\sim \operatorname{Beta}(a,b)$ when $a$ is very close to 0 while $b$ is fixed value greater than 1. Numeric ...
0
votes
0answers
123 views

Convergence of a Expectation Maximisation Algorithm

Consider using standard expectation maximisation to learn the parameters of a Hidden Markov Model. We can show the effect of standard expectation-maximisation on the log-likelihood by first writing ...
2
votes
1answer
131 views

Rate of convergence of gradient descent inference in likelihood maximization

I am reading this classic paper on convergence properties of EM for Gaussian Mixture Models. In section 5, the authors compare EM with a gradient based inference approach. The gradient approach ...
0
votes
2answers
79 views

Convergence of EM algorithm

I am aware that EM eventually converges. However, I still have some confusions regarding this property: 1: As far as I am aware, HMM, Gaussian mixture model and MCMC can converge and all of them use ...
2
votes
0answers
110 views

A comparison of the global optimal binwidth and local optimal binwidth of the histogram estimator

Suppose we have $X_1, \dots, X_n$ to be an i.i.d sample with unknown pdf $f(x)$ and cdf $F(x)$, and define $\hat{f} (x)$ to be the histogram estimator. We also define its Mean Integrated Square Error ...
0
votes
0answers
32 views

invertibility of $AR(\infty)$?

Here it writes: "Pure AR models are always invertible (since they contain no MA terms)." Is this valid also for the limiting case, that is to say, is $AR(\infty)$ invertible? Why or why not? If ...
8
votes
1answer
290 views

Limiting distribution of $\frac1n \sum_{k=1}^{n}|S_{k-1}|(X_k^2 - 1)$ where $X_k$ are i.i.d standard normal

Let $(X_n)$ be a sequence of i.i.d $\mathcal N(0,1)$ random variables. Define $S_0=0$ and $S_n=\sum_{k=1}^n X_k$ for $n\geq 1$. Find the limiting distribution of $$\frac1n \sum_{k=1}^{n}|S_{k-1}|(X_k^...
2
votes
1answer
174 views

Intuition of the convergence of sample ACF

One of the problems in Brockwell and Davis book about time series is to show that 1) if \begin{equation} x_t = a + b t \end{equation} then the sample autocorrelation ($\hat{\rho}(h)$) converges to ...
0
votes
1answer
29 views

How to prove absolute summabilities implies the absolute summability of the product series?

In SHUMWAY 2017 Time Series Analysis and Its Applications with R examples 4E, page 486, it states: $\Sigma_{j=-\infty}^{\infty} |a_j| < \infty$ and $\Sigma_{j=-\infty}^{\infty} |b_j| < \infty$ ...
1
vote
1answer
181 views

Why does absolutely-summable weights ensures a linear series itself summable (convergent)? Some questions on def'n of Linear Series

A "linear series" $y_t$ is the linear combination $$y_t - \mu = \sum_{i=-\infty}^{\infty}\psi_iL^i\nu_t = \sum_{i=-\infty}^{\infty}\psi_i\nu_{t-i}=S(L)\nu_t $$ of weighted (by $\psi_i$ weights) lags ...
3
votes
0answers
232 views

Convergence rate of the maximum of Weibull random variables to a Gumbel distribution

Given a sequence of iid samples $X_1, \dots, X_n,$ where each $X_i$ comes from a Weibull distribution with shape parameter $k$ and scale parameter $\lambda$. Then it is a well-known result that the ...
0
votes
0answers
117 views

Convergence of empirical quantiles to theoretical quantiles - mixed type distribution

It is well known that under certain (not too restrictive) conditions empirical quantiles of a distribution converge to the corresponding theoretical quantiles in probability as the sample size $n \to \...
0
votes
1answer
209 views

If $y_t$ is a time series with autocovariance $\gamma$, does $\gamma$ necessarily have to be absolutely-summable?

If $y_t$ is a time series with autocovariance $\gamma$, does $\gamma$ necessarily have to be absolutely-summable; i.e., ${\sum_{i=\infty}^\infty |\gamma (i)}|<\infty$? If not, what could be the ...
3
votes
0answers
56 views

Beneficial dimension for 2nd order modelling in SGD optimization?

There are currently mostly used first order methods in SGD optimizers, second order are often seen too costly as e.g. full Hessian has size $D^2$ in dimension $D$. But we don't need full Hessian - ...
2
votes
1answer
66 views

What is the virtue of loading absolutely-summability in the definition of causality of ARMA model?

An ARMA series $y_t$ is causal function of $\nu_t$ if there exists constants $\psi_j$ such that $\sum_{j=0}^{\infty} |\psi_j|<\infty$ and $y_t=\sum_{j=0}^{\infty} \psi_j\nu_{t-j}<\infty$ for ...
0
votes
0answers
190 views

Monte Carlo simulation percentile convergence

Suppose I have a normal distribution random variable $X$ with mean $\mu$ and variance $\sigma^2$. I can easily calculate the 99.7 percentile, which $\approx\mu+3\sigma$. In the Monte Carlo simulation ...
2
votes
1answer
36 views

Is there any statistical test to confirm if a dependent variable converges to a value as the independent variable approaches infinity?

I have a very large table with 40,000 elements, and the dependent variable appears to approach a value as the independent variable gets larger. Is there any test I can perform to confirm if the ...
1
vote
0answers
15 views

Convergence in law of the remedian

I try to understand the theroem 2 of this article about the remedian (https://pdfs.semanticscholar.org/3d64/5e60691838bf4699e79458d96930ba7bf24e.pdf) I will try to phrase it more general so that you ...
2
votes
1answer
38 views

Is this series divergent?

I have $$G_N = \sum_{i=1}^{N} \left\{ \frac{1-\pi_i}{\pi_i} + \frac{1-\pi_i}{T\pi^2_i}\right\} (y_i-\theta)^2=\sum_{i=1}^{N} V_i$$ where $2\le T\le 10$, $0\le \pi_i\le1$ and suppose $y_i\sim N(\theta,...
2
votes
1answer
392 views

Low effective sample size but good R-hat is this a problem?

I am using Stan (Hamiltonian Monte-Carlo) to run a highly paramaterized model. One of the parameters in particular has a very low effective sample size (n_eff < .10*number of retained draws), but ...
-1
votes
1answer
53 views

MCMC with slowly varying Log-Likelihood

I am using MCMC (Metropolis-Hastings) to simulate values of $\theta$: I have a Log-likelihood (using 10 inputs $x_i$) $$L=-\frac{n}{2}\ln(2\pi)-\frac{1}{2}\sum_{i=1}^n(x_i-\theta)^2$$ The variation ...
3
votes
1answer
250 views

Why is a Gelman-Rubin diagnostic of < 1.1 considered acceptable?

In multiple sources a Gelman-Rubin MCMC convergence diagnostic of less than 1.1 is considered evidence that chains have converged. For example in this thread: https://stackoverflow.com/questions/...
0
votes
0answers
14 views

Matrix inversion and $\mathcal{O}_p$ notation: $(\mathbf{A} + \mathcal{O}_p(f(n)))^{-1} = \mathbf{A}^{-1} + \mathcal{O}_p(??)$

Suppose that $\mathbf{A}$ is square invertible matrix and that $\hat{\mathbf{A}}$ is an estimator of $\mathbf{A}$ based on a sample of size $n$ such that: $\hat{\mathbf{A}}$ is invertible, $\hat{\...
2
votes
2answers
211 views

Convergence in probability does not imply convergence in $r^{th}$ mean

I am confused regarding convergence in probability and convergence in $r^{th}$ mean. I am able to prove that convergence in $r^{th}$ mean implies convergence in probability, which is not true. Let me ...
1
vote
0answers
77 views

rate of convergence acceptance-rejection vs inverse transform sampling

Does the acceptance-rejection method or inverse transform sampling converge to the mean quicker say for beta distribution, assuming acceptance-rejection has suitable envelope function? is there a way ...
1
vote
1answer
110 views

Show that $nX_{(1)}$ is not consistent

Consider a random sample from exponential distribution with mean $\frac{1}{\theta}$. I have to prove that $nX_{(1)}$ is not consistent for $\frac{1}{\theta}$ . A sufficient condition for consistency ...
1
vote
0answers
91 views

Can you use nAGQ = 0 with the Poisson distribution?

I am working with a GLM with lots of random variables and Poisson distribution. I get the error 'boundary (singular) fit: see ?isSingular' and so looked up ways around this. I found someone ...
0
votes
0answers
48 views

Limiting Distribution of $n\left[Y_n\right]$ where $Y_n$ is the minimum of a sample of size n from Uniform$\left(0,\theta\right)$ distribution

Suppose $X_1,X_2,\dots,X_n$ is a random sample from Uniform$(0,\theta)$ for some unknown $\theta > 0$. Let $Y_n$ be the minimum of $X_1,X_2,\dots,X_n$. (a) Suppose $F_n$ is the CDF of $nY_n$. Show ...
1
vote
0answers
20 views

Is it interesting to do several updates using the same batch in Stochastic Gradient Descent

I am working on a reinforcement learning problem. I was given a code where people used to train their neural-network as a Q-function estimator. During the training process, they sample $m$ (m small) ...
3
votes
0answers
44 views

Convergence rate of the inverse covariance matrix [closed]

I am trying to find results regarding the convergence rate of the inverse covariance matrix in the case where the number of observations $n$ is larger than the number of dimensions $p$. Assume that $...
1
vote
1answer
85 views

random walk on Z towards the origin

Consider a random walk on $\mathbb{Z}$ with rate $a>0$ (begin no origin). The r.w. jumps one step towards the origin with probability $p$ or one step away from the origin with probability $1 −p$. ...
2
votes
0answers
36 views

Showing $Y_n\stackrel{p}\to Z$ where $Y_n=B_nZ+(1-B_n)X$

I am reviewing some of my old class notes again, and I came across the following problem. I think I have solved the problem correctly, but I wanted to see what others here thought. Do you think I ...
1
vote
1answer
26 views

Same Example for Two Counter Examples

I'm learning some probability theory and I've come across the following: For an example of a sequence of random vairables that converges in the mean square sense but not almost surely: We set $$P(X_n=...
0
votes
0answers
122 views

Linear Mixed Model Failing to Converge

I am attempting to run a Multilevel Mediation in R with overtime data (4 time points, 50 participants). I was hoping to create two new columns for each outcome and predictor variable, a baseline ...
1
vote
0answers
243 views

Convergence of covariance matrix

I was looking for a simple way to find the number of samples $n$ needed to get a decent approximation to the covariance matrix $\boldsymbol{\Sigma}$. Given a random sample $\{ \mathbf{X}_1,\mathbf{X}...

1 2
3
4 5
16