Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

0
votes
0answers
9 views

How to quantify rate of convergence in terms of number-of-observations instead of iterations?

I observe discrete points of data, and wish to compute an integral across those points. Since the data is quite sparse, I need to interpolate and extrapolate. There are various approaches in use (...
0
votes
0answers
20 views

Posterior convergence in expectation vs probability

Let's assume that we are doing approximate Bayesian inference and compute the convergence of our posterior estimate to the true value of the parameter using Wasserstein distance. Why posterior ...
1
vote
1answer
61 views

random walk on Z towards the origin

Consider a random walk on $\mathbb{Z}$ with rate $a>0$ (begin no origin). The r.w. jumps one step towards the origin with probability $p$ or one step away from the origin with probability $1 −p$. ...
2
votes
0answers
34 views

Showing $Y_n\stackrel{p}\to Z$ where $Y_n=B_nZ+(1-B_n)X$

I am reviewing some of my old class notes again, and I came across the following problem. I think I have solved the problem correctly, but I wanted to see what others here thought. Do you think I ...
0
votes
0answers
10 views

Regression coefficient convergence

Why does the term underlined in RED converge to the term underlined in GREEN? Can someone please provide a proof? Thanks in advance!
1
vote
1answer
19 views

Same Example for Two Counter Examples

I'm learning some probability theory and I've come across the following: For an example of a sequence of random vairables that converges in the mean square sense but not almost surely: We set $$P(X_n=...
0
votes
0answers
19 views

Linear Mixed Model Failing to Converge

I am attempting to run a Multilevel Mediation in R with overtime data (4 time points, 50 participants). I was hoping to create two new columns for each outcome and predictor variable, a baseline ...
1
vote
0answers
29 views

Convergence of covariance matrix

I was looking for a simple way to find the number of samples $n$ needed to get a decent approximation to the covariance matrix $\boldsymbol{\Sigma}$. Given a random sample $\{ \mathbf{X}_1,\mathbf{X}...
1
vote
0answers
20 views

When will positive sum of random variables converge to the sum of positive ones?

Suppose we have a sequence of identically distributed continuous random variables $x_1, \dots x_N$, with $x_i \in [0,C]$, and a constant $0\leq a\leq C$. We know by Jensen's inequality that $$\...
10
votes
1answer
162 views

Is MLE of $\theta$ asymptotically normal when $(X,Y)\sim e^{-(x/\theta+\theta y)}\mathbf1_{x,y>0}$?

Suppose $(X,Y)$ has the pdf $$f_{\theta}(x,y)=e^{-(x/\theta+\theta y)}\mathbf1_{x>0,y>0}\quad,\,\theta>0$$ Density of the sample $(\mathbf X,\mathbf Y)=(X_i,Y_i)_{1\le i\le n}$ drawn from ...
1
vote
0answers
25 views

Momentum updates average of g, Adagrad also of g^2 - any other interesting updated averages for SGD convergence?

Updating exponential moving average is a basic tool of SGD methods, starting with of gradient $g$ in momentum method to extract local linear trend from the statistics. Then e.g. Adagrad, ADAM family ...
1
vote
0answers
26 views

Saddle-free Newton method for SGD - while Newton attracts saddles, is it worth to actively replel them?

While 2nd order methods have many advantages, e.g. natural gradient (e.g. in L-BFGS) attracts to close zero gradient point, which is usually saddle. Other try to pretend that our very non-convex ...
2
votes
1answer
34 views

Understanding the infinite sum of random variables

I am doing a course on time series analysis, and am struggling with this definition: We call a weakly stationary process $\{X_t\}$ invertible with respect to a white noise $\{\epsilon_t\}$ if ...
0
votes
1answer
29 views

Does it make sense to scale categorical variables in glmer when they have three levels?

I am trying to fit a generalised mixed effects model, but I am having convergence problems. The model I want to fit is ...
4
votes
1answer
49 views

The IQs from 181 boys aged between 6-7 years old were measured. Calculate its mean's confidence interval for $\alpha = 5\%$

The IQ from 181 boys aged between 6-7 years old were measured. The mean IQ is 108.08, and the standard deviation is 14.38. (a) Determine the confidence interval with confidence coefficient $95\%$ ...
0
votes
0answers
24 views

Convergence in probability (asymptotic notation) result

Let $h=h_n$ be a sequence of numbers such that $h_n \rightarrow 0$ as $n \rightarrow \infty$, $\mu$ be a real constant and $f$ be some probability density function. I was wondering if the following ...
3
votes
1answer
31 views

Distribution of N objects into C bins that are then sorted?

Let's say we have $C$ bins and $N$ indistinguishable objects. For each object we choose one bin at random where each bin is equally likely (with probability $1/C$). Let $B_k$ be the number of objects ...
2
votes
1answer
26 views

Do fully connected layers in the middle of a network impede optimization?

I submitted a paper that uses an auto-encoder network with several convolutional layers in both the encoder and the decoder and a fully connected layer (FCL) in between. Besides the FCL being useful ...
0
votes
0answers
16 views

The limit distribution of Wilcoxon signed rank statistic?

An alternative representation of the Wilcoxon signed rank statistic $V$ is $V=\sum_{i\le j}\mathbb{I}_{\{X_i+X_j>0\}}=\sum_i\mathbb{I}_{\{X_i>0\}}+\sum_{i<j}\mathbb{I}_{\{X_i+X_j>0\}}$ ...
0
votes
1answer
58 views

Convergence issues with lme4 1.1-20 for models that converged when using earlier version of lme4

I am encountering convergence problems with some models after updating to lme4 1.1-20 that I did not encounter with earlier versions of lme4 (in particular, lme4 1.1-15). I am encountering these new ...
0
votes
0answers
46 views
1
vote
0answers
14 views

Should one track the loss or accuracy of a neural network when training it?

Should one track a model's progress using its loss or its accuracy? I ask this because sometimes the loss at a epoch is higher than that at previous epochs (which is a bad thing) but so is the ...
0
votes
0answers
11 views

Are uniform LLNs preserved under monotone transformations

A simple question that I'm trying to answer is the following if I have a uniform LLN for a sequence of random vector; namely \begin{equation} \sup_\beta \left\| \frac{1}{n} \sum_n^NX_n(\beta)\right\| ...
1
vote
0answers
17 views

Is it the case/is there a proof that the convergence in distribution for the CLT is monotonic?

So for instance, if I compare $\bar{x}_n$ and the comparable normal distribution, and $\bar{x}_m$, $m > n$, and the comparable normal distribution, would I expect the difference in former (e.g. the ...
0
votes
0answers
27 views

Bayesian model initial values impact posterior values

I using Winbugs and having trouble getting the model to converge, but I think real question is understanding what is going on with the bayesian model and the initial values which is why I post on ...
0
votes
1answer
32 views
2
votes
1answer
36 views

What is a good aproximation in asymptotic normality?

I have a conceptual doubt. For example, suppose I have $X_i \stackrel{iid}{\sim} N(\theta^*,1)$ and I know that (I have the information) $\theta^{*}\geq 0$. So I have the Constrained Maximum ...
5
votes
1answer
44 views

What does the distribution of samples from an MCMC method converge to without repeated samples?

Suppose I have an absolutely continuous distribution with density $f(x)$ and I use an mcmc sampler which has accept/reject step to sample from this distribution. In the final samples, there are some ...
0
votes
0answers
34 views

Derivation of AMISE and Bandwidth

Given: Let $K(\cdot)$ be a bona fide kernel. Let $f$ be a pdf and $\widehat{f}_n$ is kernel density estimator with bandwidth $h$ based on a sample $X_1,X_2,\cdots,X_n$ of size $n$ draw iid from $f$. ...
2
votes
1answer
85 views

R: singular convergence in mixed effect model

I have an experiment that is designed as 6 blocks of 4 plots each, with two treatments (W_add and P_add) plus combination of treatments and control. The data are flux measurements taken during 9 ...
0
votes
1answer
22 views

Limits and constraints for Q-learning

I have simple implementation of Q-learning algorithm and I'm trying to run it on States space size = 36865 Actions space size = 25 So my resulting Q-table is ...
0
votes
0answers
30 views

On the Relationship between Data Size, Number of Epochs, Number of Iterations and Convergence of a Model

I did the following two experiments with a model on a dataset: Experiment 1: Training on a small dataset (~50 examples) The model took around 60 epochs to overfit just this small dataset. Each epoch ...
0
votes
1answer
28 views

Convergence in Distribution, Argument Converging in Probability

Suppose $\lim_{n\to\infty}P(X_{n}\leq x) = P(X\leq x)$ and that $A_{n} \stackrel{p}{\longrightarrow} a$, where $a$ is a continuity point of $F_{X}(x) = P(X\leq x)$. Is it the case that $\lim_{n\to\...
1
vote
0answers
30 views

Clarification regarding proof of convergence of online EM

Online EM algorithm was proposed by Olivier Cappé in Link to paper. They assume that complete data likelihood $f(x ; \theta)$ belongs to exponential family i.e. $f(x;\theta) = h(x) \exp \left\lbrace ...
1
vote
1answer
25 views

Difference between finiteness and boundedness of a random variable

In a stochastic processes class, we're studying a theorem which required that a random variable $T$ have finite mean. The notes presented a counterexample where a R.V. $T$ was such that $P(T<\infty)...
8
votes
3answers
135 views

When does $X_n\stackrel{d}{\rightarrow}X$ and $Y_n\stackrel{d}{\rightarrow}Y$ imply $X_n+Y_n\stackrel{d}{\rightarrow}X+Y$?

The question: $X_n\stackrel{d}{\rightarrow}X$ and $Y_n\stackrel{d}{\rightarrow}Y \stackrel{?}{\implies} X_n+Y_n\stackrel{d}{\rightarrow}X+Y$ I know that this does not hold in general; Slutsky's ...
5
votes
1answer
454 views

Why second order SGD convergence methods are unpopular for deep learning?

It seems that, especially for deep learning, there are dominating very simple methods for optimizing SGD convergence like ADAM - nice overview: http://ruder.io/optimizing-gradient-descent/ They trace ...
1
vote
0answers
20 views

Acceptance-Rejection using Functional

Setup Let $X\in L^1(\Omega,\mathcal{F},\mathbb{P})$. As far as I've seen, Monte-Carlo methods generate $x_1,\dots,x_n$ from the distribution of $X$ and uses the Glivenko-Cantelli theorem to conclude ...
2
votes
2answers
41 views

Asymptotic Expectation of Ratio of Sample Averages

I have two random variables: $X$ and $Y$. I know that: \begin{equation} E[X]=E[Y]=\mu>0 \end{equation} I know that variance of both can be bounded: \begin{equation} \operatorname{Var}[X]<k, \...
4
votes
1answer
49 views

Centering in longitudinal linear mixed modeling - center by participant mean, timepoint mean, or participant by time grand mean?

EDIT: I was incorrectly looking to center my outcome variables. Only center predictors, and decide on group mean or grand mean centering by how you want to interpret your intercept. I have 150 ...
0
votes
1answer
32 views

Prove convergence of a sum of random variables

I am trying to grab on to some intuition about the area where random variables start looking a bit more like calculus. I've learned about random variables and the weak law of large numbers, but seem ...
0
votes
0answers
16 views

Spatio-tempral Bayesian Poisson model convergence investigation

I am fitting a spatio-temporal Bayesian Poisson model with 22 explanatory variables, an offset variable, 2200 observations and non-informative priors. I am using the package ...
1
vote
1answer
31 views

Random variables - proof of convergence in probability

I've got this exercise from lecture notes, but I couldn't find an answer. For each positive integer $n$, let $X_{n}$ be a non-negative random variable with $\mathbb{E}[X_{n}] < \infty$. Prove that ...
1
vote
2answers
35 views

Question about expectation in OLS?

Consider the linear model $$y_i = x_i^T\beta + \epsilon_i.$$ In ordinary least squares it is assumed that the errors satisfy $E[\epsilon_i]=0$. This implies that that $\dfrac{X^T\epsilon}{n} \to 0$ ...
2
votes
1answer
192 views

abusing convergence in distribution notation

If I have $\sqrt{n} (X_n - c) \xrightarrow[]{d} N(0,v) $ does it make any sense at all to say this implies that $X_n \xrightarrow[]{d} N(c, \frac{v}{n})$. If not, what is the accurate way/notation ...
1
vote
0answers
21 views

Approximating AR(1) by finite order MA process - convergence results

I am currently struggling with a result pertaining to the finite order MA approximation of a simple AR$\,(\,1\,)$ process defined on a double sided time-index set $\,T=\mathbb{Z}$. I would be very ...
1
vote
0answers
14 views

Brownian Motion proof: difference converging to 0 almost surely

I am reading a proof where it is assumed that $$ \lim_{n \to \infty} \sup_{0<s\leq s_0}\left| \frac{t_n(s)}{s}-1 \right|=0 , \hspace{30mm} (1)$$ where $t_n(.)$ is some sequence of functions. ...
0
votes
2answers
193 views

statsmodels logistic regression with binned variables has large coefficients and standard error for some variables

I'm fitting a logistic regression (binary) using Python's statsmodels, and here's a snippet of summary from the model: I have noticed that the large coefficients ...
0
votes
1answer
84 views

glmer model convergence question

We are working with a longitudinal dataset, with three variables: WAIP, BPSRRI and group. WAIP and BPSRRI are measured repeatedly for 10 times and group refers to the group assignment of our subjects ...
0
votes
0answers
16 views

Cannot seem to converge beyond a loss of 3 on an object detector being trained on YOLO

Data The you only look once YOLO algorithm is used for object detection. I have scoured the internet for resources on how to tackle this problem, but there seems to be a lot of resources that point ...