Questions tagged [law-of-large-numbers]

Several theorems stating that sample mean converges to the expected value as $n\to\infty$. There is a weak law and a strong law of large numbers.

Filter by
Sorted by
Tagged with
1 vote
0 answers
11 views

Does the Law of Large Numbers work better for some Distributions?

Here are two popular principles in Statistics: 1) Law of Large Numbers: If $X$ is a random variable with a probability density function $f(x)$ and an expected value $E[X] = \mu$. If we take a sample ...
firstpassage's user avatar
2 votes
1 answer
35 views

Timeseries problem with law of large numbers

Let us have an AR(1) model with individual efect $$y_t = \alpha + \theta y_{t-1} + \varepsilon_t$$ with $|\theta|<1$ for stacionarity and $\varepsilon_i$ i.i.d. from distribution with mean $0$ and ...
David Paleček's user avatar
1 vote
0 answers
46 views

When is a function of an ergodic stationary process itself ergodic stationary?

I am working with a function which has the form $f(X_1, \dots, X_n)$, where $\\{X_n\\}$ is an ergodic stationary process. Theorem 5.6 in "A first course in stochastic processes" by Karlin &...
Kristan's user avatar
  • 11
1 vote
0 answers
23 views

Convergence in probability and boundness in probability with respect to sample mean and sample variance

This is a question about the convergence in probability and boundness in probability. Suppose $X_i \overset{\textrm{i.i.d.}}{\sim} (\mu, \sigma^2 )$ for $i=1,2, \cdots, n$. Denote $\overline{X}$ and $\...
d8g3n1v9's user avatar
  • 111
5 votes
3 answers
484 views

Show that $\frac{1}{n(n-1)} \sum_{i\neq j} \sin\left(X_i X_j\right)$ converges almost surely to a constant

Let $X_i$ be iid random variables. How does one show that $$ \frac{1}{n(n-1)}\sum_{i\neq j}^n \sin\left(X_i X_j\right) $$ converges almost surely to a constant?
Yashaswi Mohanty's user avatar
1 vote
1 answer
77 views

Large sample distributions

Suppose we have observations $x_1, x_2, \ldots, x_n$ where $n$ is very large. Now we standardize the observations as $$y_i=\frac{x_i-\bar{x}}{\frac{s}{\sqrt{n}}},$$ where $s=\frac{\sum\limits_{i=1}^n(...
user671269's user avatar
1 vote
0 answers
36 views

Convergence of estimated Survival Functions

Q1 part A&B I have so far $$\underset{n\rightarrow\infty} {\lim} \frac{1}{n}\sum_{i=1}^nI(T_i>x)$$ since we are summing an indicator variable we can say it has a Bernoulli distribution with ...
laxfan1212's user avatar
2 votes
1 answer
97 views

Is the Law of Large Numbers Related to the Occurrence of Rare Events Over Many Trials?

I recently watched an episode of "The Big Bang Theory" where Sheldon makes a comment about the Law of Large Numbers. In the episode, Sheldon realizes he needs eggs, and almost immediately ...
Daria's user avatar
  • 23
3 votes
1 answer
69 views

Probability Theory. Moving from finite to continuous

This has probably been asked before, as this is (I think) a fundamental theory of statistical theory, but I don't know what it is called, hence I have not yet found an answer. Consider a box which ...
user3728501's user avatar
3 votes
2 answers
94 views

What is the rigorous justification for applying LLN or CLT to finite probability spaces?

Both CLT and LLN are stated in terms of a fixed probability space that admits an infinite sequence of IID RVs. It is a common-place in many probability and statistics texts/notes that such a sequence ...
ac1501's user avatar
  • 105
0 votes
0 answers
27 views

Khinchin's Law for Variance

I know that Khinchin's Law is for sample mean: $$\overline X_n=\cfrac 1n\sum_{i=1}^{n}{X_i}$$ $$\lim _{n \rightarrow \infty} \operatorname{Pr}\{\left|\overline{X}_n-\mu\right|<\varepsilon\}=1$$ How ...
chaosink's user avatar
  • 111
3 votes
2 answers
120 views

Asymptotic normality in central limit theorem

I am a bit confused by Classical CLT section of the central limit theorem on Wikipedia. It basically says at the sample size gets larger, the difference between the sample mean and true mean ...
Sam's user avatar
  • 363
2 votes
1 answer
120 views

What are the minimum conditions needed for the consistency of OLS estimator in the following linear regression model?

Suppose $Y_i=X_i'\beta+\epsilon_i$ with $E(\epsilon_i|X_i)=0$. Consider the usual OLS estimator for $\beta$ using a random sample $\{X_i,Y_i\}_{i=1}^n$: $\widehat{\beta}=(\frac{1}{n}\sum_{i=1}^nX_iX_i'...
ExcitedSnail's user avatar
  • 2,616
1 vote
0 answers
31 views

Density weighted Law of Large Numbers argument for the convergence of an expectation approximation

Given a set of IID samples $X = \{x_i\}_{i=1}^n$ assumed to be from the density $p(\cdot)$, and the function $h:\mathbb{R} \xrightarrow{}\mathbb{R}$, its expectation can be approximated as $$\mathbb{E}...
1809's user avatar
  • 11
1 vote
0 answers
46 views

Determining the number of trees in a random forest model a priori fitting

The answer to "Do we have to tune the number of trees in a random forest?" suggests using as large a number of trees in a forest as possible. Is there a rule-of-thumb for choosing this "...
fool's user avatar
  • 2,440
1 vote
0 answers
38 views

If sample average converges a.s. in an iid sample, must it converge to the mean?

SLLN tells us that if $X_1,...,X_n$ are iid, with $X_1$ having finite mean $\mu$, then their sample average converges almost surely to $\mu$. Suppose instead we know that $X_1,...,X_n$ are iid and ...
Golden_Ratio's user avatar
2 votes
1 answer
86 views

If sample average converges in an iid sample, must it converge to the mean?

WLLN tells us that if $X_1,...,X_n$ are iid, with $X_1$ having finite mean $\mu$, then their sample average converges in probability to $\mu$. Suppose instead we know that $X_1,...,X_n$ are iid and ...
Golden_Ratio's user avatar
0 votes
0 answers
78 views

Averaging ECDF vertically : Proof of convergence

Suppose we have a set A , we split into multiple disjoint subsets ai We only have access to the ai sets , is there a way to compute the ECDF for the set A without looking at it ? If for example we ...
Amine boujida's user avatar
7 votes
2 answers
1k views

Weak law vs strong law of large numbers - intuition

I was wondering if my intuition behind the weak law (WLLN) and strong law of large numbers (SLLN) is correct. The WLLN says that, if you consider a sequence $X_1,X_2,...$,of $i.i.d.$ random variables ...
John M.'s user avatar
  • 301
6 votes
3 answers
231 views

Central limit theorem notation

Can you tell me if my understanding of the CLT is correct? Maybe it's just a matter of notation. The classical CLT states: Let $X_1,...,X_i,...,X_n$ be a sequence of iid random variables drawn from a ...
John M.'s user avatar
  • 301
1 vote
0 answers
162 views

asymptotic normality of Generalized Least Square

I am kind of new to this matrix notation and properties so I would like to see the algebraic part of the solution it helps to understand so I appreciate your understanding. My question is basically: ...
Tatanik501's user avatar
0 votes
0 answers
35 views

Scaling outliers in a dataset and reverse scaling

I have a data set with lots of small integer values and occasional large integers. For instance 1,1,1,3,2,1,320,2,3,4. I would like to scale my outlier values such that I can perform regression on my ...
murage kibicho's user avatar
2 votes
1 answer
153 views

Law of large numbers: other formulae

Let $X_1, X_2, \ldots $ be an infinite sequence of i.i.d. random variables with $E(X_i)=\mu$ and $\mbox{Var}(X_i) < \infty$. The law of large numbers states $\lim_{n \rightarrow \infty} \sum_{i=1}^{...
GCru's user avatar
  • 225
0 votes
0 answers
343 views

Law of large numbers and standard deviation

In wikipedia, the law of large numbers is defined as follows : "The average of the results obtained from a large number of trials should be close to the expected value and tends to become closer ...
HYL's user avatar
  • 355
1 vote
0 answers
255 views

How does the law of large numbers relate to regression to the mean?

Intuitively, these two important statistical principles appear to describe two facets of the same phenomenon, namely that in the long run, any extreme occurrences get counter-balanced, and things tend ...
z8080's user avatar
  • 2,300
2 votes
0 answers
27 views

Does coin tossing obey mean reversion? [duplicate]

If a coin comes up Heads 90 times out of the first 100 tosses, should one expect Tails to make a comeback over the next 100? Reference from this page: https://www.financialwisdomforum.org/gummy-stuff/...
Ehsan's user avatar
  • 21
9 votes
2 answers
223 views

Range of $a$ such that $w \leftarrow w-a x \langle w, x \rangle$ converges almost surely?

Edit Sep 19 this answer on Mathoverflow matches simulation results Suppose $x_i$ come from 2-d standard Normal centered at 0. What is the range of $a$ for which the following iteration converges ...
Yaroslav Bulatov's user avatar
0 votes
0 answers
88 views

Doesn't the CLT contain the law of large numbers?

Im sorry for asking a newbie question. The Central limit Theorem (CLT) states that when sample size tends to infinity, the sample mean will be normally distributed, and the variance is decreasing ($\...
user900476's user avatar
1 vote
0 answers
43 views

Best approximation for the size of a test

Let $X \sim \mathrm{Bernoulli}(\vartheta)$ for some unknown $\vartheta \in (0,1)$, and let $(X_1, …, X_n)$ be a moderately large IID sample for $X$. Let $\vartheta_0 \in (0,1)$. I want to test $H_0 \...
Federico's user avatar
  • 131
1 vote
0 answers
445 views

Does convergence of $\sqrt{n}X_n$ to $N(0,1)$ in distribution implies $X_n \rightarrow 0$ in probability?

This question stems from the WLLN and the Central Limit Theorem. Suppose we have $n$ iid random samples $X_1,\ldots,X_n$ with common mean $\mu$ and finite variance $\sigma^2$. Then the sample mean $\...
kkkkk's user avatar
  • 131
0 votes
0 answers
60 views

Law of large numbers for transformed and non-transformed random variables

Law of large numbers states that: If $X_1, \dots X_n \sim p(x)$ are IID, then $ \frac{1}{n} \sum_{i=1}^{n} X_i \rightarrow \mathbb{E}_{p(x)}\{X\}= \mu$, where $X \sim p(x)$. Below is what I'm having ...
Aditya Mehrotra's user avatar
2 votes
0 answers
40 views

Strong Law of Large Numbers related proof

I am trying to prove the following: So far I have used Kronecker's Lemma as such: \begin{equation} \tag{1} \text{Since } \sum_{i=1}^{\infty} \frac{\sigma_i ^2}{B_i ^2} < \infty, \text{ then, } \...
Amanda_Sterling's user avatar
0 votes
0 answers
26 views

Probability of future independent events given past and expected distribution with board games

This question is specifically in reference to how statistics apply to Catan. For background, two fair dice are rolled and rolling a 7 is a bad event. Given that the average number of turns for a game ...
wya's user avatar
  • 1
0 votes
0 answers
42 views

Using the WLLN for deriving an OLS estimator

I am reading an introductory econometrics book and I am having trouble understanding how they "directly applied the law of large numbers". Basically, they consider the case of simple linear ...
kuchejdatomas's user avatar
1 vote
0 answers
94 views

Limit of the sum of independent identically distributed random variables

The question I'm working on says: Let $X_1, X_2, \cdots$ be iid random variables each with mean $\mu$ and variance $\sigma^2$. a) Determine $$ \lim\limits_{n \to \infty} \frac{X_1^2 + \cdots + X_n^2}{...
russloewe's user avatar
33 votes
5 answers
4k views

Why should the frequency of heads in a coin toss converge to anything at all?

Suppose we have any kind of coin. Why should the relative frequency of getting a heads converge to any value at all? One answer is that this is simply what we empirically observe this to be the case, ...
Maximal Ideal's user avatar
1 vote
0 answers
34 views

$L^2$ convergence of inverse

Let $h$ be some bounded non-negative function. Assume that some random quantity $\mu^N (h)$ be some random quantity with almost sure limit $\mu(h) > 0$. For instance we could have $\mu^N(h) = N^{-1}...
yprobnoob's user avatar
  • 141
0 votes
1 answer
41 views

Is this point estimate for mean biased?

I was wondering if this point estimate for mean: $\frac{1}{n+1}\sum_{i = 1}^{n}x_i$ is biased? My first thought was that $\frac{1}{n+1}\sum_{i = 1}^{n}x_i \neq \frac{1}{n}\sum_{i = 1}^{n}x_i$, so then ...
Satan Lucifer's user avatar
20 votes
4 answers
2k views

Law of Large Numbers for whole distributions

I'm aware of the Law(s) of Large Numbers, concerning the means. However, intuitively, I'd expect not just the mean, but also the observed relative frequencies (or the histogram, if we have a ...
Igor F.'s user avatar
  • 8,817
0 votes
1 answer
81 views

The law of large number of minimum i.i.d variables random

Note that from https://projecteuclid.org/journals/annals-of-probability/volume-21/issue-3/Limit-of-the-Smallest-Eigenvalue-of-a-Large-Dimensional-Sample/10.1214/aop/1176989118.full Let $\{x_{ij}\;|\;i,...
Atina Husnaqilati's user avatar
0 votes
0 answers
114 views

The law of large numbers and coin flip probability

I really appreciate some help with the mathematics behind an example I came across: Suppose a fair coin flip where you are getting a 50% chance for heads. If you want to be 98% certain to make a ...
Dienne's user avatar
  • 1
6 votes
2 answers
882 views

Central limit theorem and strong law of large numbers

I had a question in my mind , if a i.i.d distribution function follows central limit theorem , does that mean it will follow Strong law of large numbers also ?? Since in both cases sample means tends ...
simran's user avatar
  • 377
0 votes
0 answers
64 views

Convergence to law of large numbers without using borel cantelli lemma

I am studying convergence in probability ... almost sure , in mean , And I have not studied borel cantelli lemma . Now in most of the answers on such questions this lemma is quoted . Since it is ...
simran's user avatar
  • 377
0 votes
0 answers
263 views

MGF of sample mean of poisson distribution

Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$ Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$ Then, by Weak Law of Large Numbers ...
abhishek's user avatar
  • 236
1 vote
1 answer
60 views

How can I edit this code to determine the accuracy of data? [closed]

My employer has asked me to perform a few analysis on a data about wood piles which contains their diameter and bark thickness. I am a beginner in R and started with some basic descriptive analysis ...
KimChiya's user avatar
0 votes
0 answers
43 views

The law of large numbers when using the midrange estimator for expectation

I have a growing sequence $A_1 \subseteq A_2 \cdots$ of finite sets that contain samples of $X \sim U([a, b])$. From this sequence I construct a sequence of estimates of $\mathbb{E}[X]$ using the ...
corazza's user avatar
  • 131
1 vote
1 answer
114 views

Invertibility of $E(X^\top X)$ and $\tilde{X}^\top \tilde{X}$

Consider the linear regression model with 3 regressors $$ Y=\beta_1 Q+\beta_2 W+\beta_3 Z+\epsilon $$ Let $X\equiv (Q, W, Z)$ and $\beta\equiv (\beta_1,\beta_2,\beta_3)^\top$. Also suppose that $Q,W,...
Star's user avatar
  • 816
2 votes
1 answer
319 views

What does "Expectation with respect to true unknown parameter" mean?

I am trying to study the asymptotic properties of MLE, but I am having trouble understanding an expression that seems to be consistently used in all lecture notes available online (page 93,page 18,...
CWC's user avatar
  • 281
0 votes
0 answers
135 views

What causes probability distributions to equalize?

To my understanding: Flipping a coin has a discrete 1/2 probability to be heads or tails, and every iteration of that trial resets the probability back to 1/2. So, it could be heads every time, or, ...
spyter's user avatar
  • 101
1 vote
1 answer
155 views

strong law of large numbers for V statistic

Recently, I encountered a problem regarding the a.s.-limit of $\frac{1}{n^2} \sum_{k, \ell = 1}^n ||\boldsymbol X_k - \boldsymbol X_\ell||_2$, where $\boldsymbol X_i$ are $i.i.d.$ sample following ...
Ethan's user avatar
  • 33