Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
1
vote
0answers
25 views

Does $E[\hat \Sigma^{-1}] \to \Sigma^{-1}$ still hold for samples drawn from a non-normal population?

For a sample of observations $\{x_i\}_{i=1}^n$ where $x_i=(x_{i1},\dots,x_{ik})^T$ of a population random vector $X=(X_1,\dots,X_k)^T$, the population covariance is $$ \Sigma = E[(X-E[X])(X-E[X])^T], $...
1
vote
1answer
186 views

Bounding the uniform deviation of the empirical risk from the risk over a finite function class

I am having difficulty interpreting the following theorem from here as a probability statement: Theorem. For all $\delta$ such that $0 < \delta < 1/2$, with proability at least $1 - \delta$ the ...
3
votes
1answer
56 views

Martingale property & limiting distribution for frequency of last names

Suppose that children always inherit their last names from their father (which implies that no new last names are ever created). Pick a last name of interest (e.g. Smith), and let $X_n \in \left[0, 1\...
1
vote
2answers
53 views

Intuition About Gradient Descent Convergence

I know that gradient descent takes steps towards a minimum, but I am having trouble coming up with intuitions about when it will converge. For example, on any given convex function is gradient descent ...
18
votes
2answers
1k views

Optimizing OLS with Newton's Method

Can ordinary least squares regression be solved with Newton's method? If so, how many steps would be required to achieve convergence? I know that Newton's method works on twice differentiable ...
0
votes
0answers
18 views

Convergence of the first moment of empirical distribution

Imagine I have a sequence of random variables $\{x_n\}_{n=1}^{\infty}$. These are not i.i.d. random variables, but an arbitrary sequence. For any $n$, I can define the empirical CDF function $F_n(t) = ...
0
votes
0answers
38 views

Convergence in distribution versus convergence of moments

Suppose we have that a random variable sequence $(X_n)_n$ converges in distribution to a law with mean $\bar{\mu}$ and variance $\bar{\sigma}^2$, or formally $X_n \stackrel{d}{\to} \mathcal{L}(\bar{\...
1
vote
0answers
21 views

Stochastic gradient descent convergence rate

I need to understand the convergence rate notation in the convex optimization context. In every paper that I find, the convergence rate of an algorithm is defined as a function of the number of ...
3
votes
1answer
52 views

How does using PCA speed up supervised learning?

In his popular course, Andrew Ng mentions using PCA to speed up supervised learning (Lecture 14.7). The basic idea is dimensionality reduction, wherein the extremely high-dimensional input features $\{...
0
votes
0answers
6 views

asset buying, wealth calculation, almost sure convergence

At the beginning of each year, you can buy assets for $1$ unit of money that are worth $a$ unit of money at the end of the year or stocks that are worth a random amount $v\ge 0$. If you always invest ...
1
vote
1answer
39 views

Does $X_n = O_P(a_n)$ and $a_n \to 0$ imply $X_n \stackrel{a.s.}{\to} 0$?

My attempt at this is: $$X_n = O_P(a_n) \implies P(|X_n| > C a_n) < \epsilon$$ for some $0 < C < \infty$ Then taking the limit inside the probability, we get $$P(\lim_{n \to \infty} |X_n| &...
1
vote
1answer
81 views

Proving the convergence of the maximum of Uniform Distribution

I have a random sample of size $X_1, X_2, .., X_n$ following $U(0,2)$. I need to prove that $X_{(n)}$ which is the maximum ordered statistics will converge to $2$ in probability and almost surely. I ...
0
votes
0answers
35 views

MCMC beginner question at an example chain plot: Do I need more steps? How much burn-in do I need, if I can tell already?

I am using the emcee python library to fit a model to data via MCMC. Below an example plot for the chain of one of my parameters. Here I ran 1000 steps with 100 walkers. Now I have two beginner ...
5
votes
1answer
161 views

Does the MLE converge in mean-square?

Simple question: does the MLE of a (finite-dimensional) parameter converge in mean square to the true value, that is, $$\mathbb E[\Vert\hat\theta_\text{MLE} - \theta\Vert^2_2]\rightarrow 0.$$ I know ...
6
votes
2answers
154 views

If $X_n - \mu = O_p(a_n)$ does that imply that $X_n^{-1} - \mu^{-1} = O_p(a_n)$?

If a random variable $X_n$ converges in probability to a constant $\mu$, we know by the rules for probability limits that its inverse converges to the inverse of the constant, i.e. $X_n^{-1} \stackrel{...
1
vote
0answers
46 views

Showing convergence to a (non-degenerate) distribution [closed]

The random variables $X_1$, ..., $X_n$ are i.i.d with density \begin{equation} f(x) = \begin{cases} \frac{24}{x^4}, & \text{if}\ x\geq 2 \\ 0, & \text{if x < 2} \end{cases} \...
5
votes
1answer
62 views

Can this problem really be solved using central limit theorem?

My friend had this question on a test: Let $\{X_n\}_{n \in N}$ be a sequence of independent random variables with the same normal distribution $N(0, 2n)$. Check for the convergence of a sequence $\{...
0
votes
0answers
9 views

Consistency of a simple Bayes classifier

Let $p(x,y)$ be the joint distribution of random variables $X$ and $Y$ where $$ \begin{aligned} Y&\sim \operatorname{Bernoulli}(\pi),\\ X\mid Y=y&\sim N(\mu_y,\sigma_y^2). \end{aligned} $$ Let ...
0
votes
0answers
12 views

$\limsup$ in proof that $X_n = o_p(Y_n)$ and $Y_n = O_p(1)$ then $X_n = o_p(1)$

In the proof where we have $$ P(|X_n| \geq \varepsilon) \leq P\left(|\frac{X_n}{Y_n}| \geq \frac{\varepsilon}{B}\right) + P(|Y_n| > B) $$ why do we need to take the $\limsup$ to show $X_n = o_p(1)$ ...
0
votes
0answers
39 views

What is the limiting distribution of $\chi_r^2$ random variable, where $r\to 0^+$

What is the limiting distribution of $\chi_r^2$(Chi-square) random variable, where $r\to 0^+$. The following picture shows that as $r\to 0^+$ the distribution become degenerated in zero point. If it ...
0
votes
0answers
7 views

Would very small stratum be a problem for ANCOVA model?

I know that for logistic regression, if you get an empty cell, the model may not run at all. How about continuous outcome? If one of the categorical predictors has very small stratum, would there be a ...
0
votes
0answers
43 views

Convergence of sum of (1) random variable that converges in distribution to Normal and (2) degenerate random variable that diverges to infinity?

Say that we have $\sqrt{n}(\hat{\mu} - \mu_0)$, which we can equivalently write as $\sqrt{n}(\hat{\mu} - \mu) + \sqrt{n}(\mu - \mu_0)$, where $\mu$ is the population mean, $\hat{\mu}$ is the sample ...
1
vote
0answers
20 views

Order of convergence of a product of two convergent sequences

Let $a_n$ be a sequence that converges to $A$ with order of $n^\alpha$, that is $a_n = A + \mathcal{O}(n^\alpha)$ and $b_n$ is another sequence that converges to B with order of $n^\beta$; i.e. $b_n = ...
0
votes
1answer
53 views

ML estimator of $\theta>0$ is $\hat{\theta}_n=\frac{\sum_{i=1}^nX_i^2}{n}$ and $I(\theta)=\frac{1}{\theta^2}$. Show $\hat{\theta}_n$ is consistent

In this problem we have that $I(\theta)=\frac{1}{\theta^2}$ is the Fisher information for a general probability density function $f(x;\theta)$ and $X_1,..., X_n$ are IID random variables from this ...
1
vote
0answers
18 views

Mean-square convergence of maximum likelihood estimators: Examples?

From what I've gleaned from the literature, Cràmer, in his 1947 monograph Methods of Mathematical Statistics, proved convergence in probability of an MLE under certain regularity conditions. ...
2
votes
0answers
30 views

Slutsky's theorem applied to a sample mean conditional on a Bernoulli variable?

Let $(Z_{i},Y_{2i})$, $i=1,2,\ldots,N$ be iid random vectors, where $Y_{2i}$ is the outcome vector and $Z_{i}\sim\operatorname{Bernoulli}(\delta)$. Assume that $E(Y_{2i}|Z_{i})=\mu_{2}+\beta Z_{i}$ ...
2
votes
1answer
62 views

Show $X_n \to 0$ in probability

I am asked to show : Let $X$ be a real-valued random variable on $(\Omega, F , P)$ and define $X_n(\omega) = nX(\omega)$ if $n<X(\omega)\le n+1$ and $0$ if else. Prove that $X_n \to 0$ in ...
1
vote
1answer
51 views

Is the limit in probablity of an inverse matrix equal to the inverse of the limit in probability of the matrix?

Suppose $X_n$ is a random matrix, which converges in probability to a matrix of constants, $Y$. It seems intuitive that therefore $X_n^{-1} \xrightarrow{p} Y^{-1}$ - so the limit in probability of an ...
0
votes
1answer
17 views

From Conditional Statement to an Inequality

I'm not sure about a step in the proof for the following theorem: Let $a\in \mathbb{R},$ $\{X_n\}$ be a sequence of random variables, and $g$ be a real-valued function that is continuous at $a$. ...
3
votes
1answer
20 views

Implications of zero limiting variance

Assume that I have a sequence of random variables $X_1, X_2, \dots$ with means $\mu_1, \mu_2, \dots$ such that $\lim_{n \to \infty} \operatorname{Var}(X_n) = 0$. Can I claim that for large enough $n$ ...
1
vote
1answer
114 views

Confused about conditions of the weak and strong laws of large numbers

I am a little confused by what conditions need to hold for the weak law of large numbers (WLLN) and the strong law of large numbers (SLLN) to be true. It seems different sources give me different ...
0
votes
0answers
5 views

Reference: Convergence rates for ordinal ranking

I'm looking for results on convergence rates for ordinal ranking. In particular, I'm interested in how fast the ordinal ranking loss converges to the Bayes ordinal ranking loss under various ...
2
votes
0answers
31 views

Confusion about the delta method

I'm reading Statistical Models by A. C. Davison and I'm really confused by this section on the Delta method. It's not mentioned explicitly, but is $h(T_n)$ a consistent estimator of $h(\mu)$? In the ...
4
votes
1answer
89 views

Does the law of total probability apply to hazards?

Consider the hazard function for a random variable $T$, conditional on some other random variable $U$: $$ h(t|U=u)=\lim_{\Delta t\rightarrow0}\frac{P(t<T<t+\Delta t|T>t,U=u)}{\Delta t} $$ ...
1
vote
2answers
74 views

Does the Central Limit Theorem imply that $(\hat{X}_n - \bar{x}) = o_p(1)$ at rate $O_p(1/\sqrt{n})$?

Let $\left\{\hat{X}_n\right\}$ be a sequence of estimators that converges in probability to the constant $\bar{x}$, i.e., $\left(\hat{X}_n - \bar{x}\right) = o_p(1)$. Then say that, by some applicable ...
1
vote
0answers
18 views

Forcing the mean of predictions to converge to true value

I am training a neural network on a regression problem, where I have 12 long timeseries of data that correspond to unique target values. I chop up these timeseries in very short blocks, so that I have ...
3
votes
0answers
64 views

What conditions are needed for $a_n = O_p(n^d) \implies E[a_n] = O(n^d)$?

Let $X_n$ be a uniformly integrable sequence of random variables. In a recent question I asked about the possibility of converting Big $O_p$ convergence in probability of the sequence $X_n$ to Big $O$ ...
6
votes
1answer
148 views

Can we go from $X_n = \mu + O_p(n^{-1})$ to $E[X_n] = \mu + O(n^{-1})$?

Let $X_n$ be a uniformly integrable (UI) sequence of random variables. If we have $$ X_n = \mu + O_p(n^{-1}), $$ then for $0 \le \delta < 1$ this implies $$ X_n = \mu + o_p(n^{-\delta}) \quad \quad ...
0
votes
0answers
23 views

Can I be sure about the parameter's SIGN if the convergence is problematic?

I fitted a large multilevel SEM model, used a 1 million iterations and 8 chains, but for some parameters the estimates still diverge across chains, and the corresponding autocorrelations are above .6 ...
2
votes
1answer
102 views

Rate of convergence of $\hat Q_{xx}^{-1} = \left(\frac{\mathbf{X}^T \mathbf{X}}{n}\right)^{-1}$ to the probability limit?

Consider the simple linear regression model. $$ y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \quad \quad \quad \quad i = 1,2,\dots,n. $$ Let $\mu_x$ and $\sigma_x^2$ represent the mean and variance of ...
0
votes
0answers
12 views

Numerically validating rates of convergence of approximations of expectations?

In applied mathematics it is standard practise to often validate theoretical approximations using numerical simulations. Since these simulations typically use numerical methods that convergence very ...
7
votes
2answers
100 views

Let $X,X_1,X_2,X_3,…$ be positive integer random variables. Show that $X_n \overset{d}{\to} X$ implies $\lim_{n\to\infty} P(X_n=k) = P(X=k)$

Question Let $X,X_1,X_2,X_3,...$ be positive integer random variables. Show that $X_n \overset{d}{\to} X$ implies $\lim_{n\to\infty} P(X_n=k) = P(X=k)$. The $\overset{d}{\to}$ denotes convergence in ...
3
votes
2answers
55 views

What is difference between $\hat{X}_n \overset{p}{\to} \bar{x}$ and $(\hat{X}_n - \bar{x}) = o_p(1)$?

Let $\{\hat{X}_n\}$ be a sequence of estimators that converges in probability to the constant $\bar{x}$, which I take to mean that, for any $\epsilon > 0$, $\lim \limits_{n \to \infty} \Pr(|\hat{X}...
0
votes
1answer
32 views

Proving convergence in probability

I'm looking at an example here: https://www.math.ucdavis.edu/~romik/teaching-pages/mat235a-2013/discussion8.pdf (2nd example below Lemma 5) We have that $X_n$ converges in probability to 0. I'm ...
1
vote
0answers
37 views

Where does $X_n$ converge to?

Let $ X_1, X_2, X_3, \ldots $ be independent random variables and let $ X_n $ have a probability density fucntion (PDF) defined by $ f_{X_n}(x) \quad=\quad \frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\...
0
votes
0answers
23 views

Convergence of Normalised Sum of IID Random Variables

I have a Markov chain X that starts from the stationary distribution. Let define $S_n = X_1 + \cdots + X_n.$, where $X_i$ is the state of the Markov chain. Let's have 3 states. I wanted to prove the ...
0
votes
1answer
111 views

Cross-validation not converging on Cox PH due to dummies

I'm running a Cox PH model on Python using lifelines. For some categoric variables (like purpose of loan) my go-to approach was to make it into dummies and use n-1 ...
3
votes
1answer
45 views

Z transform of $ 2^{-|n|} $

Dears, I'm trying to compute the Z-transform of $$ x(n) = 2^{-|n|} $$. My procedure is as follows: Using definition of Z transform: $$ X(z) = \sum_{n=-\infty}^{\infty}2^{-|n|} z^{-n} = \sum_{n=-\infty}...
0
votes
0answers
42 views

If a random variable $Y$ converges in distribution, can we use the parameters of the asymptotic distribution as if they are the parameters of $Y$?

Let $Y_n$ be a sequence of random variable such that $$ \sqrt{n}(Y_n-\mu) \stackrel{d}{\to} \mathcal{N}(0, \sigma^2), $$ and thus we can say $Y_n$ is asymptotically normally distributed as $$ Y_n \...
0
votes
0answers
28 views

If $X_n-X=o_p(N^{-\alpha})$, $f(\cdot)$ is smooth, do we have $f(X_n)-f(X)=o_p(N^{-\alpha})$?

If $X_n-X=o_p(N^{-\alpha})$ with $\alpha>0$, $f(\cdot)$ is smooth, do we have $f(X_n)-f(X)=o_p(N^{-\alpha})$? I guess this is true as $f(X_n)-f(X)=f'(X)(X_n-X)+o_p((X_n-X))$, which has the same ...

1
2
3 4 5
19