Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Almost sure convergence of Bernoulli random variable

Let $X_i \sim Ber(p_i)$ and independent. Prove that if $X \longrightarrow0$ $a.s.$, we have $\sum_n p_n<\infty$. My plan is to use the fact the $X \longrightarrow0$ $a.s.$ then $P(|X_n|>\...
3
votes
1answer
212 views

Convergence of Poisson Random Variable

For $n \in N $, if $X_n \sim Poisson(\frac{1}{n})$ then PT: 1. $X_n \xrightarrow[n\rightarrow \infty]{P} 0 $ $nX_n \xrightarrow[n\rightarrow \infty]{P} 0 $ It says $X_n$ converges to 0 in ...
2
votes
0answers
24 views

Cross-correlations in digit distributions

Let $B_1, B_2,\cdots$ be i.i.d. Bernouilli with mean $\frac{1}{2}$, and $$X=\sum_{k=1}^\infty \frac{B_k}{2^k}.$$ The random variables $B_k$ are the binary digits of the random number $X \in [0,1]$. ...
3
votes
2answers
120 views

How to show that quadratic mean convergence implies expectation value?

I am reading Larry Wasserman's All of Statistics and exercise 2 in chapter 6 asks for a proof that given sequence of random variables $ X_1, X_2, \dots $, show that $ X \xrightarrow{\text{QM}} b $ if ...
2
votes
1answer
77 views

how to prove almost surely

If I have $X_i$ being iid, and $E(X_i)=\infty$, how do I show that $\limsup \frac{X_n}{n}=\infty$ almost surely? I.e. how do I show $P(\limsup_{n\to\infty} (\frac{X_n}{n})= \infty)= 1$?
2
votes
1answer
151 views

Asymptotic normality for nonsmooth objective functions

Assume that $f ({\bf x}; \theta): \mathbb{R}^p \times \Theta \to \mathbb{R}$, where ${\bf x}$ is the vector of inputs (with some distribution) and $\theta$ is the vector of parameters. Also, assume ...
11
votes
2answers
582 views

What happens to the likelihood ratio as more and more data is gathered?

Let $f$, $g$ and $h$ be densities and suppose you have $x_i \sim h$, $i \in \mathbb{N}$. What happens to the likelihood ratio $$ \prod_{i=1}^n \frac{f(x_i)}{g(x_i)} $$ as $n \rightarrow \infty$ ? (...
1
vote
2answers
2k views

How do I interpret my validation and training loss curve if there is a large difference between the two which closes in sharply

my CNN is meant to classify an image as one out of around 30 categories. I am training on 6400 samples using a batch size of 128. I am using Keras/ Tensorflow Architecture is Conv + Batch ...
1
vote
1answer
460 views

SPSS: GLMM and(adjusted) odds ratio

I am performing a retrospective study and the relative statistic analysis. I am studying the the risk factors for the occurrence of complications during medical procedures. I have 50 subjects ...
0
votes
1answer
22 views

Do we have guarantees about Adam's convergence when we reach an region with gradient $0$?

Recall the Adam update rule: $$m_t = \beta_1 m_{t-1} + (1 - \beta_1) g_t$$ $$v_t = \beta_2 v_{t-1} + (1 - \beta_2) g_t^2$$ $$\hat{m}_t = \dfrac{m_t}{1 - \beta^t_1}$$ $$\hat{v}_t = \dfrac{v_t}{1 - \...
9
votes
0answers
260 views

MLE, regularity conditions, finite and infinite parameter spaces

The problem I have is in figuring out why the MLE is no longer consistent in countable parameter spaces under conditions specified below. The set up is as follows: we are consider a parameters space ...
1
vote
2answers
398 views

convergence in probability example in Casella-Berger

In Casella-Berger Statistical Inference page 234 Example 5.5.8, they define a sequence of uniform random variables $X_1, X_2, \cdots, X_n, \cdots$ such that $X_i \sim U(0,1)$ and $s \in [0,1]$ and : $...
4
votes
3answers
83 views

Limit of $t$-distribution as $n$ goes to infinity

I found in my intro to stats textbook that $t$-distribution approaches the standard normal as $n$ goes to infinity. The textbook gives the density for $t$-distribution as follows, $$f(t)=\frac{\Gamma\...
1
vote
0answers
24 views

Time series DGP: No convergence to true parameter - Identification problem?

I set up a model, simulated some data and tried to infer the wanted parameter $\alpha$. However it seems that there may be no convergence to the true parameter (result is either $-\alpha$ or $+\alpha$)...
4
votes
1answer
44 views

Is a singular fit with no correlations near +/- 1 or variances of zero, a false positive?

I sometimes get a "singular fit" warning when fitting mixed models, yet when I inspect the variance-covariance matrix of random effects, there are no correlations near -1 or +1, nor any standard ...
2
votes
2answers
137 views

Convergence in probability does not imply convergence in $r^{th}$ mean

I am confused regarding convergence in probability and convergence in $r^{th}$ mean. I am able to prove that convergence in $r^{th}$ mean implies convergence in probability, which is not true. Let me ...
1
vote
0answers
15 views

Convergence time for LSTM and Vanilla feed-forward NN training/validation errors

While learning myself, I am doing a simple example of traffic forecasting with LSTM, comparing with vanilla feedforward NN (FFNN). I observed the following When I have a large number of training ...
3
votes
1answer
40 views

How to prove convergence in probablity

Let $Y_1$, $Y_2$, ... be a sequence of random variables such that $P(Y_n=\frac{1}{n})=1-\frac{1}{n^2}$ and $P(Y_n=n)=\frac{1}{n^2}$. Does $Y_n$ converge in probability? I am stuck because I don't ...
2
votes
0answers
165 views

Why does this sequence of random variables converge in distribution?

Given iid random variables $X_1, \dots, X_n$ with common density: $$ f(x) = 1\{ x > 0 \} \cdot \frac{1}{(x+1)^2} $$ it is supposed to be the case that $\frac{\max_i X_i}{n}$ converges in ...
2
votes
0answers
22 views

Convergence Rate of Quantile from Convergence Rate of Distribution

Suppose we have a sequence of distribution functions $F_n$ such that $F_n \rightarrow F$ as $n \rightarrow \infty$. Suppose that we know the convergence rate, given by $$ \sup_{x \in \mathbb{R}} | F(...
1
vote
0answers
16 views

Continuous Mapping Theorem in infinite dimension

We have from the Continuous Mapping Theorem (CMT) that for $(X_n)_{n \in \mathbb N}$ a sequence of random variables $X_n:\Omega \to \mathbb R^p, \forall n \in \mathbb N$ and a continuous function $g: ...
0
votes
0answers
15 views

Convergence in distribution: proof strategy verification (asymptotic normality)

Suppose that $X, Y$ are random variables. My aim is to show that $X\overset{d}{\to} N(0,\sigma^2)$. If I assume that $X-Y=o_p(1)$ and $Y\overset{d}{\to} N(0,\sigma^2)$, is it right to conclude that $$...
5
votes
1answer
76 views

Does $X\stackrel{d}\to X_1$ and $Y\stackrel{d}\to Y_1$ imply $X+Y\stackrel{d}\to X_1+Y_1$?

Let $X,X_1, Y, Y_1$ be random variables. If $X\to X_1$ and $Y\to Y_1$ converge in distribution, does $X+Y\to X_1+Y_1$ in distribution?
0
votes
0answers
15 views

Convergence of Diffusion Process Monte-Carlo

Let $X_t$ be a $d$-dimensional diffusion process initialized at $x \in \mathbb{R}^d$; given as the strong solution to the SDE $$ X_t = x + \int_0^t a(t,X_t)dt + \int_0^t b(t,X_t)dW_t; $$ where $a$ and ...
7
votes
1answer
196 views

Is there a stronger Universal Approximation Theorem for LSTMs?

The Universal Approximation Theorem says that under certain conditions on your activation function, you can approximate any bounded continuous function with a feedforward neural network. I believe ...
0
votes
0answers
38 views

Convergence in distribution of a sequence indexed by a random variable

Let $(X_n(\theta))_{n \geq 1}$ be a sequence of random variables with value in $\mathbb{R}^q$ indexed by a parameter $\theta \in \Theta \subset \mathbb{R}^q$. Suppose that for all $\theta \in \Theta$: ...
2
votes
0answers
33 views

Can we use Gelman-Rubin diagnostic to assess convergence of parallel tempered chains in MCMC?

I know that the principle behind the Gelman-Rubin diagnostic is comparing within-chain and between-chain variances and if the potential scale reduction factor is less than, say 1.1 or 1.05 then the ...
1
vote
1answer
3k views

A sequence of random variables, how to understand it in the convergence theory?

I am a bit confused when studying the convergence of random variables. All the material I read using $X_i, i=1:n$ to denote a sequence of random variables. Based on the theory, a random variable is a ...
7
votes
1answer
439 views

Proof of Approximate / Exact Bayesian Computation

The ABC algorithm is given as Draw $\theta \sim \pi(\theta)$ Simulate data $X \sim \pi(x | \theta)$ Accept $\theta$ if $\rho(X, D) < \varepsilon$ where $\pi(\theta)$ is the prior, $\pi(x | \...
0
votes
0answers
20 views

Model loss stays the same for hours before dropping

I'm training a CNN to colorize images. The model I have is not incredibly deep, and should work fine on the card I'm training on (2080 TI). Initially, I suspected the model was flawed in some way ...
1
vote
1answer
64 views

Convergence of random variables problem

I am trying to solve the problem from MIT Open Coursware "Statistics for Applications" problem set. Specifically the first one: "For $n \in N^*$, let $X_n$ be a random variable such that $P[X_n = \...
2
votes
1answer
35 views

Asymptotic normality: proof strategy

Given a estimator $\hat \theta$ of $\theta$, I want to show that $\sqrt{n}(\hat\theta -\theta-B)\to N(0,V_\theta)$ as $n\to\infty$, given that the limit $V_\theta$ exists and $B>0$ possibly ...
0
votes
1answer
29 views

question about a proof of distribution

Hi all I have a question about a proof that I don't understand, My question is about the line after "We also have that....", I don't understand how $P(\hat{\theta_n} \geq \theta -\frac{x}{n})$ ...
0
votes
0answers
9 views

Generaliazation gap plots

Could some one provide me an example for a plot of generalization gap? I have understood where the x and y are from (quite silly)
0
votes
0answers
33 views

Convergence in probability for random variables

Assuming a sequence $(X_n)$ of random variables for which $\frac{X_n - \mu}{\sigma_n}\xrightarrow[]{D}N(0,1)$ as $n\xrightarrow[]{} \infty$ where $\mu \in \mathbb{R}$ and $(\sigma_n)$ converges to ...
31
votes
9answers
7k views

Expectation of 500 coin flips after 500 realizations

I was hoping someone could provide clarity surrounding the following scenario. You are asked "What is the expected number of observed heads and tails if you flip a fair coin 1000 times". Knowing that ...
5
votes
2answers
212 views

Why convergence in probability is defined as convergence to R.V.?

Wikipedia defines convergence in probability as A sequence ${X_n}$ of random variables converges in probability towards the random variable $X$ if for all ε > 0 $$\lim_{n\rightarrow\infty} P(|X_n-X|&...
1
vote
0answers
55 views

Convergence in distribution of parameters of exponential family

I am taking a course in inference where we have to find an approximate confidence interval for a Rayleigh distributed variable. The correct answer to this question states: Since we have an ...
2
votes
1answer
64 views

Find the value of $\nu$ so that $n^\nu (1-X_{(n)})$ converges in distribution

Let $X_1, X_2, \cdots$ be iid. If $X_i \sim Beta(1,\beta)$, find the value of $\nu$ so that $n^\nu (1-X_{(n)})$ converges in distribution. My thoughts: Since $X_{(n)} \to 1$ in probability, I was ...
10
votes
4answers
3k views

Intuitive understanding of the difference between consistent and asymptotically unbiased

I am trying to to get an intuitive understanding and feel for the difference and practical difference between the term consistent and asymptotically unbiased. I know their mathematical/statistical ...
9
votes
1answer
213 views

Topologies for which the ensemble of probability distributions is complete

I have been struggling quite a bit with reconciling my intuitive understanding of probability distributions with the weird properties that almost all topologies on probability distributions possess. ...
4
votes
1answer
291 views

Variance of $Z = X_1 + X_1 X_2 + X_1 X_2 X_3 +\cdots$

Here the $X_i$'s are i.i.d. and such that convergence in distribution for the infinite sum, is guaranteed. Probably the easiest case is when $X_i$ has a Bernouilli($p$) distribution, then $Z$ has a ...
2
votes
0answers
34 views

Convergence of series of dependent random variable, central limit theorem

My friend and I have a problem on central limit theorem. Given $X_1,X_2......$ are i.i.d random variables with mean $\mu$=0, variance $\sigma^2=1$(may or may not be normally distributed). If we ...
0
votes
0answers
11 views

What is the relationship between Validation loss and Training loss when considering Overfitting? [duplicate]

Here I have results from my training stage I have been told that this would not be considered as overfitting, however, it seems the line follows the dots well and the validation loss is higher than ...
2
votes
1answer
90 views

Can importance sampling be used as an actual sampling mechanism?

This question is a duplicate of How can we use importance sampling for drawing posterior distribution samples? , but that question seems to lack additional detail and goes unanswered (for more than 2 ...
3
votes
1answer
71 views

How can I show convergence in distribution to the normal?

Consider the following model: $$Y|N \sim \mathcal{X}^2_{2N} \quad \quad \quad N \sim \text{Pois}(\theta).$$ and define the standardised statistic: $$Z = \frac{Y-\mathbb{E}(Y)}{\sqrt{\mathbb{V}(Y)}...
2
votes
0answers
40 views

How can I find the limiting distribution of $Z_n=\sqrt{n}\frac{X_1X_2+X_3X_4+\cdots+X_{2n-1}X_{2n}}{X_1^2+\cdots+X_{2n}^2}$?

Let $X_1,X_2,\cdots$ be i.i.d random variables with $E(X_i)=0$, $Var(X_i)=1$, and $E(X_i^4)<\infty$. How can I find the limiting distribution of $Z_n=\sqrt{n}\frac{X_1X_2+X_3X_4+\cdots+X_{2n-...
3
votes
1answer
81 views

What is the limiting distribution of $Y_n = \sqrt{n}(\bar{X}_n-1)$ as $n \to \infty$?

Let $X_1,\cdots,X_n$ be independently and identically distributed with pdf $f(x)=e^{-x}, 0 < x < \infty$. Let $Y_n = \sqrt{n}(\bar{X}_n-1)$. What is the limiting distribution of $Y_n$ as $n \to ...
5
votes
3answers
60 views

How can I show that $X_n=e^n I_{\{Y>n\}} \to 0$ in probability?

Let $Y$ be a continuous random variable with density function $f_Y(y)=e^{-y}, y > 0$. Consider the sequence $\{X_n\}$, given by $X_n=e^nI_{\{Y>n\}}, n =1,2,\cdots$ How can I show that $\{X_n\} \...
0
votes
0answers
39 views

Model convergence problem; non-positive-definite Hessian matrix - small variance

I want to see the differences between the 6 conditions regarding a centralization index (CI). I am trying to GLMM using the package glmmTMB in R but the following warning appears Warning messages: 1:...

1 2 3 4 5 16