Questions tagged [law-of-large-numbers]
Several theorems stating that sample mean converges to the expected value as $n\to\infty$. There is a weak law and a strong law of large numbers.
125
questions
1
vote
1answer
70 views
Limiting distribution of $\sum_{j=1}^{p}\lambda_j U_j$
Assume $U_j$ are $\chi^2(1)$ random variables and $\lambda_1, \ldots, \lambda_p$ are the eigenvalues of a covariance matrix $\Sigma = (r^{|i-j|})_{ij}$ with a Toeplitz-type structure (for some fixed $|...
4
votes
1answer
35 views
Law of Large Numbers for geometrically decaying sequences
Let $(X_n)_n$ be a sequence of i.i.d. random variables, and let $\rho \in (0,1)$. Is there any asymptotic Theorem for the following random variable:
$$
Y = \lim_{n\rightarrow \infty}\sum_{i=0}^n \rho^...
0
votes
1answer
45 views
Sum of squared normals: $ \sum_{k=1}^{n}U_k^2 = \frac{1}{n}\sum_{k=1}^{n}a_k^2 \cdot \Gamma\left(\frac{n}{2},\frac{1}{2}\right)$
Assume $U_k \sim \mathcal{N}(0,a_k^2)$, where $a_k \rightarrow c > 0$ as $k \rightarrow \infty$. It follows that $U_k^2 \sim \Gamma(\frac{1}{2}, \frac{1}{2a_k^2})$.
I'm interested in the exact and ...
0
votes
1answer
42 views
Does central limit theorem help in making inference only about the population mean and not other parameters?
As per the weak law of large numbers, if your sample size is large, your mean of the sample is likely to be closer to the population mean than in a smaller sample. Additionally the CLT tells us the ...
0
votes
1answer
37 views
Central Limit Theorem Approximation and Relation to Law of Large Numbers
Assume the Linberg-Levy CLT to where we know
$$\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma}\xrightarrow{d}N(0,1).$$
I feel like I commonly see then that
$$\bar{X}_n\approx N(\mu,\frac{\sigma^2}{n}),$$
but ...
0
votes
1answer
24 views
Weird behavior in simulating LLN for standard deviations in R
I am simulating the law of large numbers and how it applies to standard deviation, as well. I wrote this code that works well but there is something that I am having a hard time understanding.
...
1
vote
1answer
80 views
Confused about conditions of the weak and strong laws of large numbers
I am a little confused by what conditions need to hold for the weak law of large numbers (WLLN) and the strong law of large numbers (SLLN) to be true. It seems different sources give me different ...
0
votes
1answer
23 views
Terminology - population and sample in repeated rolling dice experiments
The the law of large numbers wikipedia page has an example on fair, six-sided dice:
According to the law of large numbers, if a large number of fair six-sided dice are rolled, the average of their ...
0
votes
0answers
16 views
Finite sample behavior of learning algorithms
In machine learning, given a joint distribution $\mathbb P_{X, Y}$, where $Y=\{0, 1\}$ making it a binary classification problem, and $N$ iid training samples $\{(x_i, y_i)\}_{i=1}^N$ and iid test ...
3
votes
0answers
27 views
Bernoulli distribution and Strong Law of Large Numbers
I'm trying to find concrete examples of the SLLN theorem. Before, let's see the statement of this theorem precisely from this book, page 81:
Definition: We say that $X_n$ converges almost surely to $...
6
votes
3answers
671 views
Law of Large Numbers
Suppose X is uniform discrete distribution from a set of (1,2,3,...,m). How do i investigate the law of large numbers for this? I thought of doing this by maybe setting m as 10 and having sample of ...
0
votes
0answers
21 views
Convergence of Normalised Sum of IID Random Variables
I have a Markov chain X that starts from the stationary distribution. Let define $S_n = X_1 + \cdots + X_n.$, where $X_i$ is the state of the Markov chain. Let's have 3 states. I wanted to prove the ...
0
votes
0answers
26 views
If $X_n-X=o_p(N^{-\alpha})$, $f(\cdot)$ is smooth, do we have $f(X_n)-f(X)=o_p(N^{-\alpha})$?
If $X_n-X=o_p(N^{-\alpha})$ with $\alpha>0$, $f(\cdot)$ is smooth, do we have $f(X_n)-f(X)=o_p(N^{-\alpha})$?
I guess this is true as $f(X_n)-f(X)=f'(X)(X_n-X)+o_p((X_n-X))$, which has the same ...
1
vote
0answers
19 views
Is there a law or concept related to unchanging average?
I have a survey comprising 30 questions with 50k respondents where respondents mark each answer on a 5 point scale. An average of all the questions for all the respondents is metric we track monthly. ...
1
vote
0answers
34 views
Weak Law of Large Number to Central Limit Theorem [closed]
Consider a distribution with unknown mean μ and population standard deviation σ=30. Using the Weak Law of Large Numbers, what is the minimum sample size in order to attain a probability of at least 99%...
0
votes
1answer
35 views
Law of large numbers applied to $\overline{X}$ [duplicate]
The law of large numbers states that if we have $X_1,\ldots X_n$ independent variables which are identically distributed, then:
$$\overline{X}:=\frac{1}{n}\sum_{i=1}^nX_i \longrightarrow \mu \quad \...
0
votes
1answer
62 views
Difference between uniform laws of large numbers and law of large numbers
How to understand the difference between uniform laws of large numbers and law of large numbers? In particular, what does the word "uniform" mean?
1
vote
0answers
37 views
Are this simple claim and its proof correct?
Suppose random sequence $\{X_{i}(N)\}_{i=1}^{N}$ is a row-wise i.i.d. triangular array, where $N$ is sample size. This means for any given $N$, $X_{i}(N),\dots,X_{N}(N)$ are i.i.d. following ...
1
vote
1answer
108 views
Why taking an average makes convergence to zero faster?
Let $f(x,y)$ be some density, and let the leave-one-out Nadaraya-Watson estimator $\widehat{f}_{-i}(x,y)$ be defined as follows:
$\widehat{f}_{-i}(x,y)=\frac{1}{(n-1)h^2}\sum_{j=1,j\neq i}^nK(\frac{(...
0
votes
1answer
31 views
aquiring different results for a limit using CLT and LLN
I'm having some trouble with convergence in distribution and convergence in probability, mainly because I'm getting different results that seem to contradict each other using the Central limit theorem ...
0
votes
0answers
58 views
Convergence of bootstrap standard error estimate (one of the problems from Efron's book)
I am trying to solve problem 2.5 from the Efron/Tibshirani book, An Introduction to the Bootstrap (page 16).
The problem asks to show that by applying the weak law of large numbers, the bootstrap ...
4
votes
1answer
85 views
Proving Identifiability Using Law of Large Numbers? [closed]
Well normally proving identifiability follows by showing that $p_{\theta}(x)=p_{\theta'}(x)$ implies $\theta=\theta'$. Usually this proceeds by showing that a function dependent on $\theta$, such as ...
1
vote
2answers
62 views
Does random walking have a memory?
I do not know if this is a good question, but I have not found an answer for it anywhere, does a binomial random variable have memory, [simple logic says it can have no memory, but statistics show ...
1
vote
1answer
42 views
Law of large numbers vs Short events vs Probability
:STATEMENT:
Suppose an individual plays a gambling game where it is possible to lose $1.00$, break even, win $3.00$, or win $10.00$ each time she plays. The probability distribution for each outcome ...
1
vote
0answers
35 views
Central limit theorem implications about $\bar{X}$
The central limit theorem (CLT) and law of large numbers (LLN) look to make the same claim.
$$\text{CLT: }\sqrt{n}\big(\bar{X}_n-\mu\big) \rightarrow N(0,\sigma^2) $$
$$\text{LLN: }\bar{X}_n \...
1
vote
1answer
47 views
Linear regression with one generated regressor
Suppose I have the regression model: $Y_i=T^{\top}_{i}\beta_0+e_{i}$ with $E(e_i|X_i)=0$, where we have two regressors $X_i,\ E(D|X_{i})$ so that $T^{\top}_{i}=[X_i,\ E(D|X_{i})]$. $X_{i}$ is a ...
3
votes
1answer
59 views
Can someone provide a proof for this theorem that was used in the proof of law of large number?
Let $X_1,X_2,X_3,...$ be $i.i.d.$ with finite mean $\mu$.
Then let $Y_i=X_i1_{\{|X_i|<i\}}$
There will be only finitely many terms such that $Y_i\neq X_i$
While that lecture notes did not ...
0
votes
0answers
21 views
Is this proof of convergence in probability to zero correct?
I want to show that $A=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\widehat{B}_{i}-B_{i})X_i$ converges in probability to 0, where $B_i=E(C_i|Z_i)$ and $C_i$ is i.i.d. binary and $Z_i$ is a discrete random ...
3
votes
1answer
452 views
Showing the weak law of large number of non-IID sequence of random variables
Common proofs on the law of large numbers usually assume a sequence of IID random variables. If $X_1\dots X_n$ has a common expected value $\mu$, finite but not necessarily common variance (hence not ...
1
vote
1answer
260 views
Should bootstrapping and collecting sample means from a series of binomial distributions result in standard normal?
I’m trying to better understand several statistical concepts (bootstrapping, central limit theorem, and confidence intervals) by applying them to a binomial distribution (you can think of it as a coin ...
4
votes
2answers
76 views
Why do typical sequences have probabilities $\sim2^{-nH(p)}$?
I've been reading a bit about typical sequences (in particular from these notes (pdf alert), pages 3 and 4). Let us focus on the case of binary sequences for simplicity. As far as I understand the ...
0
votes
0answers
30 views
4
votes
1answer
194 views
Posterior mean with MCMC
Let's say we have a posterior distribution:
$$\pi(\theta_1, \theta_2, \theta_3 | \bf{y})$$
and that we've run an MCMC algorithm to approximate this distribution.
I know that there is a Markov chain ...
31
votes
9answers
8k views
Expectation of 500 coin flips after 500 realizations
I was hoping someone could provide clarity surrounding the following scenario. You are asked "What is the expected number of observed heads and tails if you flip a fair coin 1000 times". Knowing that ...
0
votes
0answers
46 views
Binomial Distribution:equivalency between the average and proportion?
Another user and I are having a debate regarding properties of a binomial distribution, most notably the relationship between the proportion and the mean. The central prompt is in quotations:
“I ...
1
vote
2answers
207 views
Optimal way to estimate the irreducible error E [ V [Y | X] ]?
I was reading about conditional variance in Wikipedia and then the following property showed up
$$E[V[Y \mid X]]=E[(Y-f(X))^2]-E[(E[Y \mid X]-f(X))^2]$$
Which i interpret as an irreducible error, ...
2
votes
0answers
61 views
Beyond the CLT: guarantees on the shape of the sample mean distribution?
If the L.L.N. tell us where our sample mean is going, and the C.L.T. "extend it" telling us how fast the variance is decaying, do we have an other tool telling us how fast the shape (or the further ...
0
votes
0answers
7 views
Work flow question regarding sample theory
The question is pulled below from a book.
The solution to this problem is also posted below.
I am able to get the result before taking the expectation by applying the sum of a finite geometric ...
0
votes
1answer
72 views
Flattening of sums (Law of Large Numbers or Central Limit Theorem)
If we add $N$ curves with independent random phases, the shape of the sum tends to flatten out as $N\to \infty$. That is, the fluctuation depth decreases as $N$ gets larger. Is this phenomenon a ...
1
vote
1answer
88 views
random walk on Z towards the origin
Consider a random walk on $\mathbb{Z}$ with rate $a>0$ (begin no origin). The r.w. jumps one step towards the origin with probability $p$ or one step away from the origin with probability $1 −p$. ...
2
votes
1answer
327 views
Expressing Law of Large numbers in terms of binomial probabilities
Suppose I let $p \in (0,1)$. Then for $n \ge 1$ and $0 \le k \le n$ let $${P}(k;n,p) = \dbinom{n}{k} p^k(1-p)^{n-k}$$.
How can I express the law of large numbers in terms of the probabilities $P(k;n,...
1
vote
0answers
46 views
Acceptance-Rejection using Functional
Setup
Let $X\in L^1(\Omega,\mathcal{F},\mathbb{P})$.
As far as I've seen, Monte-Carlo methods generate $x_1,\dots,x_n$ from the distribution of $X$ and uses the Glivenko-Cantelli theorem to conclude ...
2
votes
2answers
84 views
Asymptotic Expectation of Ratio of Sample Averages
I have two random variables: $X$ and $Y$. I know that:
\begin{equation}
E[X]=E[Y]=\mu>0
\end{equation}
I know that variance of both can be bounded:
\begin{equation}
\operatorname{Var}[X]<k, \...
0
votes
1answer
67 views
Prove convergence of a sum of random variables
I am trying to grab on to some intuition about the area where random variables start looking a bit more like calculus.
I've learned about random variables and the weak law of large numbers, but seem ...
0
votes
0answers
43 views
Law of Large Numbers under Transformation
The Law of Large Number, draw i.i.d examples of a random variable y, then with propability of 1 the average of y_1, ... y_n goes to the expected value of y.
When i apply a function to the y_i's, does ...
1
vote
0answers
71 views
Residuals and the law of large numbers
Can I apply the law of large numbers (LLN) to $ê$ (the observed residuals)? For example, can I use LLN for ($1/n)∑ê$ and say that converge to $E[e]$ (where $e$ is the true error)?
I think that I can'...
1
vote
1answer
785 views
Convergence in probability of $\frac{1}{n}\sum_{i=1}^n X_i^2$ when $X_i$'s are i.i.d $N(0,1)$
Question:
My approach:
And after this I am stuck..How do I put the modulus over here and how do I determine the appropriate value of "k" ?
(here k signifies the value of convergence in probability ...
2
votes
1answer
597 views
Law of Large Numbers for Covariance Stationary Processes… Difference and Relationship between LLN and Ergodicity
We have a covariance stationary time series. We must assume that the time series was produced by an ergodic process if we are to make the bridge between the realization of the time series that we ...
0
votes
1answer
38 views
Can we conclude, with the strong law of large numbers, that $n$ random variables are independent? [closed]
Suppose we have a sequence of identically distributed random variables $X_1, \ldots, X_n$, and that we know $(X_1 + \ldots + X_n)/n$ converges almost surely to $\mu = E[X]$ as $n$ approaches infinity. ...
2
votes
1answer
406 views
Sampling from finite population with replacement
Suppose we have a population of size 10,000. We choose 1000 persons from the population, uniformly and independently, and ask them some question(s). The answers are random variables $X_i$.
Since we ...
