Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Convergence in distribution: proof strategy verification (asymptotic normality)

Suppose that $X, Y$ are random variables. My aim is to show that $X\overset{d}{\to} N(0,\sigma^2)$. If I assume that $X-Y=o_p(1)$ and $Y\overset{d}{\to} N(0,\sigma^2)$, is it right to conclude that $$...
5
votes
1answer
68 views

Does $X\stackrel{d}\to X_1$ and $Y\stackrel{d}\to Y_1$ imply $X+Y\stackrel{d}\to X_1+Y_1$?

Let $X,X_1, Y, Y_1$ be random variables. If $X\to X_1$ and $Y\to Y_1$ converge in distribution, does $X+Y\to X_1+Y_1$ in distribution?
0
votes
0answers
14 views

Convergence of Diffusion Process Monte-Carlo

Let $X_t$ be a $d$-dimensional diffusion process initialized at $x \in \mathbb{R}^d$; given as the strong solution to the SDE $$ X_t = x + \int_0^t a(t,X_t)dt + \int_0^t b(t,X_t)dW_t; $$ where $a$ and ...
7
votes
1answer
87 views

Is there a stronger Universal Approximation Theorem for LSTMs?

The Universal Approximation Theorem says that under certain conditions on your activation function, you can approximate any bounded continuous function with a feedforward neural network. I believe ...
0
votes
0answers
36 views

Convergence in distribution of a sequence indexed by a random variable

Let $(X_n(\theta))_{n \geq 1}$ be a sequence of random variables with value in $\mathbb{R}^q$ indexed by a parameter $\theta \in \Theta \subset \mathbb{R}^q$. Suppose that for all $\theta \in \Theta$: ...
2
votes
0answers
31 views

Can we use Gelman-Rubin diagnostic to assess convergence of parallel tempered chains in MCMC?

I know that the principle behind the Gelman-Rubin diagnostic is comparing within-chain and between-chain variances and if the potential scale reduction factor is less than, say 1.1 or 1.05 then the ...
0
votes
0answers
19 views

Model loss stays the same for hours before dropping

I'm training a CNN to colorize images. The model I have is not incredibly deep, and should work fine on the card I'm training on (2080 TI). Initially, I suspected the model was flawed in some way ...
2
votes
0answers
57 views
+50

How to show that quadratic mean convergence implies expectation value?

I am reading Larry Wasserman's All of Statistics and exercise 2 in chapter 6 asks for a proof that given sequence of random variables $ X_1, X_2, \dots $, show that $ X \xrightarrow{\text{QM}} b $ if ...
1
vote
1answer
63 views

Convergence of random variables problem

I am trying to solve the problem from MIT Open Coursware "Statistics for Applications" problem set. Specifically the first one: "For $n \in N^*$, let $X_n$ be a random variable such that $P[X_n = \...
0
votes
0answers
9 views

Generaliazation gap plots

Could some one provide me an example for a plot of generalization gap? I have understood where the x and y are from (quite silly)
0
votes
1answer
26 views

question about a proof of distribution

Hi all I have a question about a proof that I don't understand, My question is about the line after "We also have that....", I don't understand how $P(\hat{\theta_n} \geq \theta -\frac{x}{n})$ ...
0
votes
0answers
33 views

Convergence in probability for random variables

Assuming a sequence $(X_n)$ of random variables for which $\frac{X_n - \mu}{\sigma_n}\xrightarrow[]{D}N(0,1)$ as $n\xrightarrow[]{} \infty$ where $\mu \in \mathbb{R}$ and $(\sigma_n)$ converges to ...
0
votes
1answer
18 views

Do we have guarantees about Adam's convergence when we reach an region with gradient $0$?

Recall the Adam update rule: $$m_t = \beta_1 m_{t-1} + (1 - \beta_1) g_t$$ $$v_t = \beta_2 v_{t-1} + (1 - \beta_2) g_t^2$$ $$\hat{m}_t = \dfrac{m_t}{1 - \beta^t_1}$$ $$\hat{v}_t = \dfrac{v_t}{1 - \...
31
votes
9answers
7k views

Expectation of 500 coin flips after 500 realizations

I was hoping someone could provide clarity surrounding the following scenario. You are asked "What is the expected number of observed heads and tails if you flip a fair coin 1000 times". Knowing that ...
1
vote
0answers
50 views

Convergence in distribution of parameters of exponential family

I am taking a course in inference where we have to find an approximate confidence interval for a Rayleigh distributed variable. The correct answer to this question states: Since we have an ...
2
votes
1answer
35 views

Asymptotic normality: proof strategy

Given a estimator $\hat \theta$ of $\theta$, I want to show that $\sqrt{n}(\hat\theta -\theta-B)\to N(0,V_\theta)$ as $n\to\infty$, given that the limit $V_\theta$ exists and $B>0$ possibly ...
2
votes
1answer
61 views

Find the value of $\nu$ so that $n^\nu (1-X_{(n)})$ converges in distribution

Let $X_1, X_2, \cdots$ be iid. If $X_i \sim Beta(1,\beta)$, find the value of $\nu$ so that $n^\nu (1-X_{(n)})$ converges in distribution. My thoughts: Since $X_{(n)} \to 1$ in probability, I was ...
2
votes
0answers
30 views

Convergence of series of dependent random variable, central limit theorem

My friend and I have a problem on central limit theorem. Given $X_1,X_2......$ are i.i.d random variables with mean $\mu$=0, variance $\sigma^2=1$(may or may not be normally distributed). If we ...
0
votes
0answers
11 views

What is the relationship between Validation loss and Training loss when considering Overfitting? [duplicate]

Here I have results from my training stage I have been told that this would not be considered as overfitting, however, it seems the line follows the dots well and the validation loss is higher than ...
1
vote
0answers
38 views

How can I find the limiting distribution of $Z_n=\sqrt{n}\frac{X_1X_2+X_3X_4+\cdots+X_{2n-1}X_{2n}}{X_1^2+\cdots+X_{2n}^2}$?

Let $X_1,X_2,\cdots$ be i.i.d random variables with $E(X_i)=0$, $Var(X_i)=1$, and $E(X_i^4)<\infty$. How can I find the limiting distribution of $Z_n=\sqrt{n}\frac{X_1X_2+X_3X_4+\cdots+X_{2n-...
3
votes
1answer
73 views

What is the limiting distribution of $Y_n = \sqrt{n}(\bar{X}_n-1)$ as $n \to \infty$?

Let $X_1,\cdots,X_n$ be independently and identically distributed with pdf $f(x)=e^{-x}, 0 < x < \infty$. Let $Y_n = \sqrt{n}(\bar{X}_n-1)$. What is the limiting distribution of $Y_n$ as $n \to ...
5
votes
3answers
59 views

How can I show that $X_n=e^n I_{\{Y>n\}} \to 0$ in probability?

Let $Y$ be a continuous random variable with density function $f_Y(y)=e^{-y}, y > 0$. Consider the sequence $\{X_n\}$, given by $X_n=e^nI_{\{Y>n\}}, n =1,2,\cdots$ How can I show that $\{X_n\} \...
3
votes
1answer
68 views

How can I show convergence in distribution to the normal?

Consider the following model: $$Y|N \sim \mathcal{X}^2_{2N} \quad \quad \quad N \sim \text{Pois}(\theta).$$ and define the standardised statistic: $$Z = \frac{Y-\mathbb{E}(Y)}{\sqrt{\mathbb{V}(Y)}...
2
votes
1answer
83 views

Can importance sampling be used as an actual sampling mechanism?

This question is a duplicate of How can we use importance sampling for drawing posterior distribution samples? , but that question seems to lack additional detail and goes unanswered (for more than 2 ...
0
votes
0answers
33 views

Model convergence problem; non-positive-definite Hessian matrix - small variance

I want to see the differences between the 6 conditions regarding a centralization index (CI). I am trying to GLMM using the package glmmTMB in R but the following warning appears Warning messages: 1:...
1
vote
0answers
20 views

How does type of convergence affect results in practice? [closed]

I've been exposed to many different types of convergence of random variables, namely: convergence almost surely convergence in the $r^{th}$ moment convergence in probability convergence in ...
0
votes
0answers
16 views

Continuous mapping theorem and random vectors

My question is on whether or not continuous mapping theorem can be applied to elements of a random vector. Consider $[X_n,Y_n] \rightarrow [\mu, \sigma]$ Would it also be true that for any ...
4
votes
1answer
287 views

Variance of $Z = X_1 + X_1 X_2 + X_1 X_2 X_3 +\cdots$

Here the $X_i$'s are i.i.d. and such that convergence in distribution for the infinite sum, is guaranteed. Probably the easiest case is when $X_i$ has a Bernouilli($p$) distribution, then $Z$ has a ...
0
votes
0answers
14 views

Time Distributed Loss

I am currently working on implementing a time series prediction task that will produce labels across a sequence (batch, steps, features) -> (batch, steps, classes). I have a TimeDistributed layer as ...
1
vote
0answers
33 views

Why is Algorithm 8 of Neal (2000) a valid sampler?

I have been having difficulty understanding why Algorithm 8 of Neal (2000) is a valid sampler. I am looking for lecture notes that include a nice explanation of the proof. Does anyone know of any ...
1
vote
1answer
103 views

Model converges in glmmTMB but not lme4, why?

I am running what I suppose is the same mixed-effect model with a negative binomial distribution (log link) in both lme4 and the glmmTMB package in R. Code shown below: ...
2
votes
1answer
54 views

Understanding convergence in Bayesian inference of coin tossing

When we are uncertain about the probability of head, $p_H$, in a coin tossing, we often model it using a Beta prior as follows: $$p_H\sim \text{Beta}(a_0,b_0),$$ for some parameters $a_0,b_0$. When ...
0
votes
0answers
312 views

Lasso regression doesn't converge in case of zero Y-vector

I try to use lasso regression to solve linear problem with big amount of equations (~10 000). Everything worked fine, but I noticed that if in Y-vector all elements are equal, "fit" function hang for ...
0
votes
0answers
18 views

Basic results on convergence in distribution

Let $\{X_i\}_{i=1}^n$ be independent zero mean random variables with finite variance and $\{r_n\},\{d_n\}$ positive monotone increasing real sequences. Assume that $$\frac{\sum_{i=1}^{r_n} X_i}{Var(\...
0
votes
0answers
19 views

the convergence speed for a Markov chain

For a metropolis hastings algorithm, suppose that the stationary distribution is defined as the Gibbs Boltzmann distribution $\pi_T(x)= \frac{1}{Z_T}e^{-\frac{V(x)}{T} }$ where $Z_T = \sum_{y\in V} e^{...
2
votes
1answer
30 views

Clarification about the limiting distribution and approximate distribution of $\bar{X}^3$ using the delta method

The question states: Let $X_1,....X_n$ be a random sample from $f(x$,$\theta$) with $E(X$) = $\mu$ and $V(X) = \sigma^2$. Find the limiting distribution of $\bar{X}^3$, and the approximate ...
1
vote
0answers
8 views

Bootstrap: mixing independent and time-series data together

I have a very computationally heavy simulator (large-scale agent-based transport simulation), which usually takes up to 5 days of run time in a large computer. The results are probabilistic, so ...
7
votes
2answers
664 views

Why is the limit of a Chi squared distribution a normal distribution?

My professor claimed that $\lim_{p\to\infty}\chi^2_p$ has a normal distribution. The claim was made on the basis of the Central Limit Theorem: as $p\to\infty$, we have a Normal$(p\mu, p^2\sigma^2)$. I ...
1
vote
2answers
100 views

Show that the distribution of $\frac{1}{\sqrt{n}}\sum_{i=1}^n(X_i^2-3)$ is normal

Let $X_1,\ldots,X_n$ be i.i.d. variables with $\mathbb{E}[X_i]=0$ and $\mathbb{V}[X_i]=3$ and assume that $\mathbb{E}[X^4_i]<\infty$, show that $$ \frac{1}{\sqrt{n}}\sum_{i=1}^n(X_i^2-3) $$ ...
0
votes
0answers
27 views

Which is the better estimator for standard deviation?

Let $X_i \sim^{\textrm{iid}} N(\mu, \sigma^2)$. If I have measured $n$ values of $\textrm{std}(X_i)$ as $\sigma_1,\cdots,\sigma_n$, then what is the better estimator for $\sigma$: $$\hat{\sigma}_1 = ...
0
votes
1answer
34 views

Probability of an event with probability 0 happening at least once in infinite trials

This question here is confusing me a lot. To summarize, let's say you have $\text{i.i.d. }X_i \sim U(0, 1), i = 1,2,\ldots, n.$ The question shows that $Y_i = \max(X_1, \ldots, X_i) \rightarrow 1$ ...
0
votes
1answer
31 views

How can we conclude that an optimization algorithm is better than another one for a problem at hand

When we test a new optimization algorithm for a particular problem at hand, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,...
2
votes
0answers
64 views

Proving Linear Regression with Gradient Descent Converge to OLS estimates

Problem I am having trouble showing that the parameters $\theta\in \mathbf{R}^{m}$ for Linear Regression converge to the classic OLS estimates using gradient descent. Please find below my attempt: ...
0
votes
0answers
50 views

convergence in distribution when parameters converges almost surely

let $X_n$ be sequence of random variables with the associated distribution $N(\mu_n,\beta_n+E)$. That is a sequence of normally distributed random variables with changing mean and variance. $\beta_n$ ...
0
votes
1answer
118 views

How many iterations are too many?

I have the following model: ...
0
votes
0answers
23 views

Issue with proof in Statistical Theory related to Beta and Binomial distributions [duplicate]

Assume $X_n$ is distributed $\text{Beta}(1/n, 1/n)$ and $X$ is distributed as $\text{Binom}(1,1/2)$. Show that $X_n$ converges to $X$ in distribution. I'm having some issue with this question. I ...
1
vote
0answers
20 views

convergence and efficiency of mcmc chains and estimation of covariance matrix

I am doing some bayesian analysis and exploring posterior distribution with mcmc method. I would like some clarification with estimating the covariance matrix. I have a model with 6 parameters. ...
0
votes
0answers
142 views

Time complexity of batch gradient descent

I am read http://papers.nips.cc/paper/4937-accelerating-stochastic-gradient-descent-using-predictive-variance-reduction.pdf paper. It states that "Due to the poor condition number, the standard batch ...
0
votes
1answer
42 views

Real Analysis like convergence of Loss Functions [closed]

Here's one thing i noticed. In Elementary Real Analysis, when we say that a sequence $s_n$ converges to a point $s$, we first set an $\epsilon > 0$ such that for large $N \in \mathbb{N} $ $| s_n - ...
0
votes
0answers
18 views

Uniform Convergence of Partial Autocorrelation

I've been facing the problem of estimating a large PACF as the sample size grows. My question is whether we can guarantee that the partial autocorrelation, estimated by the projection $X_t = \sum_{...