Questions tagged [convergence]

Convergence generally means that a sequence of a certain sample quantity approaches a constant as the sample size tends to infinity. Convergence is also a property of an iterative algorithm to stabilize on some aim value.

Filter by
Sorted by
Tagged with
6
votes
1answer
623 views

Issues with using Expectation Maximization algorithm

I was using the EM algorithm to maximize a partially observed likelihood. However, I have certain doubts. Normally, the algorithm works fine. I could print the value of the log likelihood of the ...
2
votes
2answers
4k views

R's coxph won't converge when I include factor (categorical) variables

I have a dataset of 371 observations. When I run coxph with numeric variables it works fine. However, when I try to add factor (categorical) variables it returns “...
1
vote
1answer
2k views

MCMC autocorrelation convergence diagnostic

I use MCMC (Metropolis-Hastings) to sample posterior distributions of three parameters using a nonlinear least-squares objective function to calculate the likelihood of a parameter sets. I want ...
1
vote
1answer
267 views

MCMC and terrible convergence

The code and data I am borrowing come from http://www.perossi.org/home/bsm-1 under CS 5 from the book Bayesian Statitics and Marketing. I tried applying their model to another dataset and am getting ...
4
votes
0answers
256 views

How to visually show the convergence of a process?

I have some set of measurements that I have represented as vectors $x^t$ for $t \in \{ 1, 2, ...\}$. I want to test "convergence" of the process (visually) in some sense. I thought maybe I could run ...
2
votes
0answers
59 views

Confusion related to the derivation of a dual of a problem

I have this confusion related to the derivation of the dual. I was referring to these lecture slides. I didn't get how the dual was derived. I didn't get how the dual was derived. I am ok up to the ...
1
vote
1answer
384 views

pLSA using tempered EM

In an article by Hofmann pdf, he proposes: initialize $β$ to one, run until convergence, then rescale $β$ by a factor $η<1$, run again until convergence, and iterate this until changing $β$ ...
9
votes
4answers
3k views

(interacting) MCMC for multimodal posterior

I am trying to sample from a posterior having many modes particularly far from each others using MCMC. It appears that in most cases, only one of these modes contains the 95% hpd I am looking for. I ...
4
votes
1answer
244 views

Convergence rate of a non-linear function of the sample mean

We have a iid sequence of random variables $X_1, X_2, \dots, X_n$, where $E(X_i) = \mu$ and $var(X_i) = \sigma^2$. The sample mean $\bar{X}$ converges to $\mu$ at rate $\sqrt{n}$ thanks to the LLN. ...
1
vote
1answer
261 views

Asymptotic normality and normalization wrt variance

Let $X_n, n \in \mathbb N$ be a sequence of random variables with finite variances. As $n \to \infty$, are the following two equivalent: $X_n \to N(0, \sigma^2)$ for some $\sigma^2 \in [0, \infty)$, $...
4
votes
1answer
1k views

Issues with estimating the sparse inverse covariance matrix with Glasso [closed]

I am trying to estimate the sparse inverse covariance matrix of my gaussian graphical model. I installed the glasso package in R and tried out some examples. After ...
3
votes
2answers
129 views

Convergence of empirical distribution parameters of a sequence of generated normal variables

I am generating a sequence of normal random variables (using the routines from boost C++ library). How fast would you expect the mean and the variance of the sequence converge to the actual variance? ...
2
votes
2answers
126 views

Condition for Law of Large Numbers, Monte Carlo

In some lecture notes I am reading, there is the following; Consider $X_{1},...,X_{n}$, each with pdf $g$ (the instrumental distribution). Our aim is to estimate $E_{f}[h(X)]$ where $h(X)$ is some ...
2
votes
1answer
2k views

How to find a sequence of random variables of infinite expectation that converges to zero?

I want to find an example such that $X_n\rightarrow0$ as $n\rightarrow \infty$ while $E(X_n)=\infty$. I am thinking of making use the divergence of $\sum\frac{1}{n}$, but fail to find suitable $X_n$ ...
1
vote
0answers
2k views

xtlogit: panel data transformation's recast to double makes model incomputable (STATA)

I am experiencing a very peculiar problem in STATA. I have a panel dataset with 8000 groups and 4 million observations. I want to run a fixed effects panel binary logistic regression (xtlogit). As ...
1
vote
2answers
19k views

Binary panel logistic regression (xtlogit fixed effects) is not converging in Stata, how to resolve?

I have a panel dataset with a sample of 800 groups, each having between 200-500 observations. The data looks like this: The dependent variable is binomial: ...
2
votes
1answer
207 views

Implication of convergence in rth mean

I'm trying to show that if $X_n \rightarrow X $ in $r$-th mean, then $E|X_n|^r \rightarrow E|X|^r$. (Edit: should have said with $r \ge 1$) My question is whether the following steps are sufficient ...
19
votes
1answer
3k views

Central limit theorem and the law of large numbers

I have a very beginner's question regarding the Central Limit Theorem (CLT): I am aware that the CLT states that a mean of i.i.d. random variables is approximately normal distributed (for $n \to \...
2
votes
0answers
481 views

On the uniform convergence of relative frequencies of events to their probabilities

I have read the article by Vapnik, Chervonenkis "On the uniform convergence of relative frequencies of events to their probabilities" Theory of Probability and Its Applications, vol XVI, n. , 1971. ...
4
votes
0answers
188 views

Best practices for MCMC early stopping?

What are best and / or standard practices for MCMC early stopping? I have an algorithm which I want to compare with existing non-MCMC algorithms for accuracy and speed. When assessing the speed it's ...
17
votes
5answers
24k views

Why doesn't k-means give the global minimum?

I read that the k-means algorithm only converges to a local minimum and not to a global minimum. Why is this? I can logically think of how initialization could affect the final clustering and there is ...
5
votes
2answers
3k views

Variables that do not converge in winbugs

I conducted a Bayesian analysis in winbugs and then checked the convergence from the history plots. The regression coefficients look stabilized, but the variance parameters don't. I got the number of ...
3
votes
1answer
1k views

Convergence in distribution and CDF

Suppose $X_n$ converges in distribution to $X$ , $x_n \rightarrow x$, also the cumulative distribution function for $X$ is continuous at $x$. Show that $ P(X_n \leq x_n) \rightarrow P(X \leq x)$. PS:...
2
votes
1answer
133 views

A problem on a.s. convergence

I'm preparing for an exam and I came across this problem from old exams. I'm really clueless on how to solve it. Consider a sequence of random variables $\{X_n\}_{n=1} ^\infty$ defined on the ...
6
votes
3answers
329 views

Find the limiting distribution of Sum over Sum of Squares

Having a little trouble with this one: Suppose $X_1, X_2, \ldots $ are iid standard normal random variables. Let $W_n = \sqrt{n} \frac{X_1 + \cdots + X_n}{X_1^2 + \cdots + X_n^2}$. Find the limiting ...
4
votes
1answer
448 views

Optimal rate of convergence for nonparametric estimators in Sobolev space

Given a regression model on interval $[0,1]$ $$ Y_{i}=f(x_{i})+\epsilon_{i},\ i=1,\ldots,N $$ with fixed design and standard error assumptions $E(\epsilon_{i})=0;\ E(\epsilon_{i}\epsilon_{j})=\delta_{...
2
votes
0answers
455 views

Time dependent weights in hidden Markov models

I'm trying to modify a standard implementation of a continuous HMM with Gaussian Mixtures so that it internally gives more weight to newer observations in a time series. Essentially, I'm trying to ...
2
votes
0answers
1k views

Negative binomial regression does not converge

I am trying to use a Negative Binomial regression model for some count data in which the dependent count variable takes on the values 0, 1, 2, 3 or 4. I am using SAS and keep running into the warning:...
2
votes
0answers
485 views

Sum of Squares reduction test: Convergence criteria met, but not all parameters of the model estimated

Background: I am using weighted non-linear regression to model the growth of plant organs, with dummy variables for different species. I am using a sum of squares reduction test (SSRT) to compare the ...
5
votes
2answers
1k views

Convergence of random variables

Trying to understand the solution given to this homework problem: Define random variables $X$ and $Y_n$ where $n=1,2\ldots%$ with probability mass functions: $$ f_X(x)=\begin{cases} \frac{1}{2} &...
6
votes
1answer
417 views

Showing that the power of a test approaches 1 as the sample size approaches infinity

I'm working on some exercises for my econometrics class and I'm a little confused. I'm meant to consider a model $$Y=\beta_0 +\beta_1X+u$$ and propose a test (test statistic and critical value) of $...
3
votes
1answer
175 views

Convergence problem

I practiced some problem (not homework) as below. Suppose that $X_1,X_2,\cdots$ are independent and identically distributed real-valued random variables with $E|X_1|=\infty$. Show that $\sum_{n=1}^\...
3
votes
1answer
467 views

Almost sure convergence

The problem (not homework) I practiced is Consider the probability space $([0,1], B_{[0,1]}, P)$ where $B_{[0,1]}$ is the Borel set and $P$ is Lebesgue measure on $[0,1]$. For any integer $n>0$, ...
1
vote
1answer
123 views

Weak Convergence

Here is the problem (not homework), Let $U_1,\cdots,U_n$ be i.i.d. uniform$(-n,n)$ random variables. For $-n<a<b<n$, we set $1_{U_i}(a,b)$ be the indicator function such that $1_{U_i}=1$ if ...
7
votes
2answers
239 views

Convergence of distribution

This is from Probability and Measure by Billingsley, 3rd Edition. 27.21 (p. 370) Let $X_1, X_2,...$ be independent and identically distributed, and suppose that the distribution common to the $X_n$ ...
7
votes
1answer
2k views

Latent variables, overparameterization and MCMC convergence in bayesian models

Sometimes I have a large number of latent variables in a Bayesian hierarchical model to which, but I am only interested in estimating projected transformations of those latent variables (for example, ...
7
votes
1answer
200 views

Laplacian-Beltrami approximation based on an empirical sample

Given a probability measure $\nu$ on a subset $M \subseteq \mathbb{R}^N$ we construct the corresponding operator $$L^tf(x)=f(x)\int_{M} e^{-\frac{||x-y||^2}{4t}}d\nu(y)-\int_{M}f(y)e^{-\frac{||x-y||^...
7
votes
1answer
136 views

Convergence of MCMC for ill-behaved functions

We are doing MCMC sampling over an n-dimensional space with a population of MCMCs. The goal is to have them stop when they approximately converge. The problem is that the convergence, which is so ...
3
votes
0answers
274 views

Proving the convergence of KDE algorithms when the samples are non-i.i.d

I am currently working on convergence proof for a new method for non-parametric importance sampling, and I need some help... My method uses an MCMC algorithm to generate a set of dependent $M$ ...
4
votes
2answers
792 views

How to check whether maximum likelihood estimation optimizer has converged in R?

I got AIC values of all models to identify the best model using R language. As I heard, best model produce the smallest AIC value, but maximum likelihood estimation procedure optimizer should converge....
3
votes
0answers
548 views

What is a “polynomially bounded” function, and why is this a requirement of the The Delta Method?

I am reading a paper "A note on the Delta Method" by Gary Oehlert, JASA, 1992. I am trying to estimate the variance of a function of a random variable, but first I want to understand the limitations ...
0
votes
1answer
180 views

Mean-reversion for random process

I moedelled following ARIMA Process which I also smoothed. abc1 <- (arima.sim(n = 1400, list(ar = c(0.3, 0.2, 0.15, 0.1, 0.1, 0.1)), sd = sqrt(0.5))) plot(abc1) abc2 <- SMA(abc1, n=360) ...
6
votes
2answers
770 views

Limit of a convolution and sum of distribution functions

I need to prove an induction step. $X_i$ are independently distributed with the distribution function $1-F_i=x^{-\alpha}L_{i}(x)$ where $\alpha \geq 0$ and $L_{i}(x)$ is regularly varying (If the ...
2
votes
1answer
391 views

Asymmetrical selective sampling for linear classification

I've got a online classification problem where I predict a class label {+1, -1} for an object and then show it to a user to get a real label. My task is to minimize ...
2
votes
2answers
2k views

How to find parameters for ridge and lasso regularization when cost minimization does not converge?

In the Stanford ML course, we were taught to find good values for the lambda parameters of ridge/lasso by iterating for various lambda values on several cross-validation sets and picking the values ...
5
votes
1answer
1k views

Markov chain convergence, total variation and KL divergence

I have a few related questions regarding the convergence of continuous-state Markov chains. The theorems that I found claim that Markov chains converge in total variation if they are $\phi$-...
6
votes
1answer
6k views

Convergence in probability and $L_2$ for normal random variables

In an answer here: Convergence of identically distributed normal random variables, the following lemma is mentioned: Lemma: Let $X_1, X_2, \ldots$ be a sequence of zero-mean normal random ...
3
votes
0answers
2k views

Convergence failure with step-by-step parameter estimates in SAS proc logistic using the OFFSET option

I'm trying to fit a logistic model of the following form: $$ y \sim \frac{1}{1 + \exp(-\beta_X)} $$ where $$ \beta_X = a_0 + a_1x_1 + a_2x_2 + a_3x_3 + a_4x_4 + a_5x_5 $$ In my case, there is ...
5
votes
1answer
1k views

Convergence in distribution, probability, and 2nd mean

Let $\mathbb P(X=1) = \mathbb P(X=-1) = 1/2$. Define $$X_n = \begin {cases} X & \text{with probability } 1- \frac{1}{n}\\ e^n & \text{with probability } \frac{1}{n} \end {cases}$$ ...
7
votes
4answers
2k views

Convergence of a genetic algorithm

Does anyone know of any method for deciding when a genetic algorithm is done? In MCMC (e.g, BUGS), several chains are started at different, random points. When they all look the same, it is done. Has ...