Questions tagged [law-of-large-numbers]

Several theorems stating that sample mean converges to the expected value as $n\to\infty$. There is a weak law and a strong law of large numbers.

Filter by
Sorted by
Tagged with
0 votes
0 answers
32 views

Finding covariance matrix for bootstrapped errors in OLS

Let's say we have matrix $x \in \mathbb{R}^{n \times k}$, $y \in \mathbb{R}^n$ and $\beta^*$ vector, which $\beta^* = \arg\min_\phi\sum_i (y_i - x_i\phi)$, i.e. we have classic regression problem and $...
user avatar
  • 201
1 vote
0 answers
20 views

Limit of the sum of independent identically distributed random variables

The question I'm working on says: Let $X_1, X_2, \cdots$ be iid random variables each with mean $\mu$ and variance $\sigma^2$. a) Determine $$ \lim\limits_{n \to \infty} \frac{X_1^2 + \cdots + X_n^2}{...
user avatar
33 votes
5 answers
4k views

Why should the frequency of heads in a coin toss converge to anything at all?

Suppose we have any kind of coin. Why should the relative frequency of getting a heads converge to any value at all? One answer is that this is simply what we empirically observe this to be the case, ...
user avatar
1 vote
0 answers
14 views

$L^2$ convergence of inverse

Let $h$ be some bounded non-negative function. Assume that some random quantity $\mu^N (h)$ be some random quantity with almost sure limit $\mu(h) > 0$. For instance we could have $\mu^N(h) = N^{-1}...
user avatar
  • 131
0 votes
0 answers
27 views

Sufficient condition for ergodicity in the first-order moment

We have a WSS process $X_n$ with expectation $\mu_X$. We assume the covariance function converges to zero as $k$ increases. I feel like its a simple question that I don't understand, If you could give ...
user avatar
0 votes
1 answer
41 views

Is this point estimate for mean biased?

I was wondering if this point estimate for mean: $\frac{1}{n+1}\sum_{i = 1}^{n}x_i$ is biased? My first thought was that $\frac{1}{n+1}\sum_{i = 1}^{n}x_i \neq \frac{1}{n}\sum_{i = 1}^{n}x_i$, so then ...
user avatar
19 votes
4 answers
1k views

Law of Large Numbers for whole distributions

I'm aware of the Law(s) of Large Numbers, concerning the means. However, intuitively, I'd expect not just the mean, but also the observed relative frequencies (or the histogram, if we have a ...
user avatar
  • 6,283
0 votes
1 answer
72 views

The law of large number of minimum i.i.d variables random

Note that from https://projecteuclid.org/journals/annals-of-probability/volume-21/issue-3/Limit-of-the-Smallest-Eigenvalue-of-a-Large-Dimensional-Sample/10.1214/aop/1176989118.full Let $\{x_{ij}\;|\;i,...
user avatar
0 votes
0 answers
54 views

The law of large numbers and coin flip probability

I really appreciate some help with the mathematics behind an example I came across: Suppose a fair coin flip where you are getting a 50% chance for heads. If you want to be 98% certain to make a ...
user avatar
  • 1
6 votes
2 answers
436 views

Central limit theorem and strong law of large numbers

I had a question in my mind , if a i.i.d distribution function follows central limit theorem , does that mean it will follow Strong law of large numbers also ?? Since in both cases sample means tends ...
user avatar
  • 437
0 votes
0 answers
26 views

Convergence to law of large numbers without using borel cantelli lemma

I am studying convergence in probability ... almost sure , in mean , And I have not studied borel cantelli lemma . Now in most of the answers on such questions this lemma is quoted . Since it is ...
user avatar
  • 437
0 votes
0 answers
81 views

MGF of sample mean of poisson distribution

Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$ Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$ Then, by Weak Law of Large Numbers ...
user avatar
  • 187
1 vote
1 answer
59 views

How can I edit this code to determine the accuracy of data? [closed]

My employer has asked me to perform a few analysis on a data about wood piles which contains their diameter and bark thickness. I am a beginner in R and started with some basic descriptive analysis ...
user avatar
  • 13
0 votes
0 answers
31 views

The law of large numbers when using the midrange estimator for expectation

I have a growing sequence $A_1 \subseteq A_2 \cdots$ of finite sets that contain samples of $X \sim U([a, b])$. From this sequence I construct a sequence of estimates of $\mathbb{E}[X]$ using the ...
user avatar
  • 131
1 vote
1 answer
75 views

Invertibility of $E(X^\top X)$ and $\tilde{X}^\top \tilde{X}$

Consider the linear regression model with 3 regressors $$ Y=\beta_1 Q+\beta_2 W+\beta_3 Z+\epsilon $$ Let $X\equiv (Q, W, Z)$ and $\beta\equiv (\beta_1,\beta_2,\beta_3)^\top$. Also suppose that $Q,W,...
user avatar
  • 276
2 votes
1 answer
132 views

What does "Expectation with respect to true unknown parameter" mean?

I am trying to study the asymptotic properties of MLE, but I am having trouble understanding an expression that seems to be consistently used in all lecture notes available online (page 93,page 18,...
user avatar
  • 261
0 votes
0 answers
98 views

What causes probability distributions to equalize?

To my understanding: Flipping a coin has a discrete 1/2 probability to be heads or tails, and every iteration of that trial resets the probability back to 1/2. So, it could be heads every time, or, ...
user avatar
  • 101
0 votes
0 answers
25 views

Multiple regression - covariance matrix, law of large numbers

I'm currently reading an article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4302277/#S6title And in Section 6 there is a least squre estimator of $\beta$ given of course as $\hat \beta = (X_c'X_c)^{...
user avatar
  • 15
1 vote
1 answer
66 views

strong law of large numbers for V statistic

Recently, I encountered a problem regarding the a.s.-limit of $\frac{1}{n^2} \sum_{k, \ell = 1}^n ||\boldsymbol X_k - \boldsymbol X_\ell||_2$, where $\boldsymbol X_i$ are $i.i.d.$ sample following ...
user avatar
2 votes
1 answer
87 views

Limiting distribution of $\sum_{j=1}^{p}\lambda_j U_j$

Assume $U_j$ are $\chi^2(1)$ random variables and $\lambda_1, \ldots, \lambda_p$ are the eigenvalues of a covariance matrix $\Sigma = (r^{|i-j|})_{ij}$ with a Toeplitz-type structure (for some fixed $|...
user avatar
  • 632
4 votes
1 answer
43 views

Law of Large Numbers for geometrically decaying sequences

Let $(X_n)_n$ be a sequence of i.i.d. random variables, and let $\rho \in (0,1)$. Is there any asymptotic Theorem for the following random variable: $$ Y = \lim_{n\rightarrow \infty}\sum_{i=0}^n \rho^...
user avatar
  • 41
0 votes
1 answer
83 views

Sum of squared normals: $ \sum_{k=1}^{n}U_k^2 = \frac{1}{n}\sum_{k=1}^{n}a_k^2 \cdot \Gamma\left(\frac{n}{2},\frac{1}{2}\right)$

Assume $U_k \sim \mathcal{N}(0,a_k^2)$, where $a_k \rightarrow c > 0$ as $k \rightarrow \infty$. It follows that $U_k^2 \sim \Gamma(\frac{1}{2}, \frac{1}{2a_k^2})$. I'm interested in the exact and ...
user avatar
  • 632
0 votes
1 answer
174 views

Does central limit theorem help in making inference only about the population mean and not other parameters?

As per the weak law of large numbers, if your sample size is large, your mean of the sample is likely to be closer to the population mean than in a smaller sample. Additionally the CLT tells us the ...
user avatar
  • 11
0 votes
1 answer
68 views

Central Limit Theorem Approximation and Relation to Law of Large Numbers

Assume the Linberg-Levy CLT to where we know $$\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma}\xrightarrow{d}N(0,1).$$ I feel like I commonly see then that $$\bar{X}_n\approx N(\mu,\frac{\sigma^2}{n}),$$ but ...
user avatar
0 votes
1 answer
37 views

Weird behavior in simulating LLN for standard deviations in R

I am simulating the law of large numbers and how it applies to standard deviation, as well. I wrote this code that works well but there is something that I am having a hard time understanding. ...
user avatar
1 vote
1 answer
368 views

Confused about conditions of the weak and strong laws of large numbers

I am a little confused by what conditions need to hold for the weak law of large numbers (WLLN) and the strong law of large numbers (SLLN) to be true. It seems different sources give me different ...
user avatar
  • 357
0 votes
1 answer
174 views

Terminology - population and sample in repeated rolling dice experiments

The the law of large numbers wikipedia page has an example on fair, six-sided dice: According to the law of large numbers, if a large number of fair six-sided dice are rolled, the average of their ...
user avatar
  • 143
3 votes
0 answers
69 views

Bernoulli distribution and Strong Law of Large Numbers

I'm trying to find concrete examples of the SLLN theorem. Before, let's see the statement of this theorem precisely from this book, page 81: Definition: We say that $X_n$ converges almost surely to $...
user avatar
  • 507
6 votes
3 answers
1k views

Law of Large Numbers

Suppose X is uniform discrete distribution from a set of (1,2,3,...,m). How do i investigate the law of large numbers for this? I thought of doing this by maybe setting m as 10 and having sample of ...
user avatar
1 vote
0 answers
19 views

Is there a law or concept related to unchanging average?

I have a survey comprising 30 questions with 50k respondents where respondents mark each answer on a 5 point scale. An average of all the questions for all the respondents is metric we track monthly. ...
user avatar
  • 41
1 vote
0 answers
72 views

Weak Law of Large Number to Central Limit Theorem [closed]

Consider a distribution with unknown mean μ and population standard deviation σ=30. Using the Weak Law of Large Numbers, what is the minimum sample size in order to attain a probability of at least 99%...
user avatar
0 votes
1 answer
40 views

Law of large numbers applied to $\overline{X}$ [duplicate]

The law of large numbers states that if we have $X_1,\ldots X_n$ independent variables which are identically distributed, then: $$\overline{X}:=\frac{1}{n}\sum_{i=1}^nX_i \longrightarrow \mu \quad \...
user avatar
  • 135
0 votes
1 answer
332 views

Difference between uniform laws of large numbers and law of large numbers

How to understand the difference between uniform laws of large numbers and law of large numbers? In particular, what does the word "uniform" mean?
user avatar
  • 153
1 vote
0 answers
37 views

Are this simple claim and its proof correct?

Suppose random sequence $\{X_{i}(N)\}_{i=1}^{N}$ is a row-wise i.i.d. triangular array, where $N$ is sample size. This means for any given $N$, $X_{i}(N),\dots,X_{N}(N)$ are i.i.d. following ...
user avatar
  • 1,682
1 vote
1 answer
126 views

Why taking an average makes convergence to zero faster?

Let $f(x,y)$ be some density, and let the leave-one-out Nadaraya-Watson estimator $\widehat{f}_{-i}(x,y)$ be defined as follows: $\widehat{f}_{-i}(x,y)=\frac{1}{(n-1)h^2}\sum_{j=1,j\neq i}^nK(\frac{(...
user avatar
  • 1,682
0 votes
1 answer
37 views

aquiring different results for a limit using CLT and LLN

I'm having some trouble with convergence in distribution and convergence in probability, mainly because I'm getting different results that seem to contradict each other using the Central limit theorem ...
user avatar
  • 123
4 votes
1 answer
92 views

Proving Identifiability Using Law of Large Numbers? [closed]

Well normally proving identifiability follows by showing that $p_{\theta}(x)=p_{\theta'}(x)$ implies $\theta=\theta'$. Usually this proceeds by showing that a function dependent on $\theta$, such as ...
user avatar
  • 1,415
1 vote
2 answers
73 views

Does random walking have a memory?

I do not know if this is a good question, but I have not found an answer for it anywhere, does a binomial random variable have memory, [simple logic says it can have no memory, but statistics show ...
user avatar
2 votes
1 answer
65 views

Law of large numbers vs Short events vs Probability

:STATEMENT: Suppose an individual plays a gambling game where it is possible to lose $1.00$, break even, win $3.00$, or win $10.00$ each time she plays. The probability distribution for each outcome ...
user avatar
1 vote
0 answers
39 views

Central limit theorem implications about $\bar{X}$

The central limit theorem (CLT) and law of large numbers (LLN) look to make the same claim. $$\text{CLT: }\sqrt{n}\big(\bar{X}_n-\mu\big) \rightarrow N(0,\sigma^2) $$ $$\text{LLN: }\bar{X}_n \...
user avatar
  • 31k
1 vote
1 answer
79 views

Linear regression with one generated regressor

Suppose I have the regression model: $Y_i=T^{\top}_{i}\beta_0+e_{i}$ with $E(e_i|X_i)=0$, where we have two regressors $X_i,\ E(D|X_{i})$ so that $T^{\top}_{i}=[X_i,\ E(D|X_{i})]$. $X_{i}$ is a ...
user avatar
  • 1,682
3 votes
1 answer
72 views

Can someone provide a proof for this theorem that was used in the proof of law of large number?

Let $X_1,X_2,X_3,...$ be $i.i.d.$ with finite mean $\mu$. Then let $Y_i=X_i1_{\{|X_i|<i\}}$ There will be only finitely many terms such that $Y_i\neq X_i$ While that lecture notes did not ...
user avatar
5 votes
1 answer
1k views

Showing the weak law of large number of non-IID sequence of random variables

Common proofs on the law of large numbers usually assume a sequence of IID random variables. If $X_1\dots X_n$ has a common expected value $\mu$, finite but not necessarily common variance (hence not ...
user avatar
  • 873
1 vote
1 answer
724 views

Should bootstrapping and collecting sample means from a series of binomial distributions result in standard normal?

I’m trying to better understand several statistical concepts (bootstrapping, central limit theorem, and confidence intervals) by applying them to a binomial distribution (you can think of it as a coin ...
user avatar
  • 1,949
4 votes
2 answers
105 views

Why do typical sequences have probabilities $\sim2^{-nH(p)}$?

I've been reading a bit about typical sequences (in particular from these notes (pdf alert), pages 3 and 4). Let us focus on the case of binary sequences for simplicity. As far as I understand the ...
user avatar
  • 225
0 votes
0 answers
31 views

Law of Large Numbers

How do i proceed with this? any hints would help
user avatar
  • 31
4 votes
1 answer
277 views

Posterior mean with MCMC

Let's say we have a posterior distribution: $$\pi(\theta_1, \theta_2, \theta_3 | \bf{y})$$ and that we've run an MCMC algorithm to approximate this distribution. I know that there is a Markov chain ...
user avatar
  • 229
31 votes
9 answers
9k views

Expectation of 500 coin flips after 500 realizations

I was hoping someone could provide clarity surrounding the following scenario. You are asked "What is the expected number of observed heads and tails if you flip a fair coin 1000 times". Knowing that ...
user avatar
  • 423
0 votes
0 answers
78 views

Binomial Distribution:equivalency between the average and proportion?

Another user and I are having a debate regarding properties of a binomial distribution, most notably the relationship between the proportion and the mean. The central prompt is in quotations: “I want ...
user avatar
1 vote
2 answers
442 views

Optimal way to estimate the irreducible error E [ V [Y | X] ]?

I was reading about conditional variance in Wikipedia and then the following property showed up $$E[V[Y \mid X]]=E[(Y-f(X))^2]-E[(E[Y \mid X]-f(X))^2]$$ Which i interpret as an irreducible error, ...
user avatar