Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
220
questions
0
votes
0
answers
22
views
$E(SN)$ for aggregate claim amount $S$, $S=X_{1}+...+X_{n}, X_{i}$ are iid [duplicate]
Consider the following model for aggregate claim amounts $S$:
$S=X_{1}+X_{2}+...+X_{N}$
where the $X_{i}$ are independent, identically distributed random variables representing individual claim ...
1
vote
1
answer
56
views
Studying extreme value r.v. $X=\max_i (c_i+X_i)$ where $c_i$ are constants and $X_i$ are i.i.d. r.v
Let
$X_1,X_2,...,X_n$ be independently and identically distributed random variables according to a distribution $F$.
There are constants: $c_1,c_2,...,c_n$.
Define a new random variable $X=\max_i(X_i+...
- 143
2
votes
0
answers
36
views
Test for distribution equality
This question touches Kolmogorov-Smirnov testing, but asks actually something different.
Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis:
$$
H_0: X_i\ ...
- 145
2
votes
1
answer
72
views
MCMC Sample should be i.i.d
I'm a bit not sure how to show that MCMC samples are i.i.d. In my opinion the trace plot should behave like white noise model because white noise model has a strong stationary properties i.e. the ...
0
votes
0
answers
41
views
If we remove half the samples from an IID dataset, is the remaining half still IID?
I am generating 10,000 pairs of X and Y such that both X and ...
- 241
0
votes
0
answers
32
views
understanding "independent" term in "independent and identically distributed" (iid) statement
I understand independent as the random variable values in a data aren't connected to each other in any way.
(q1) Is my understanding ok?
data_1: a product which has a design defect has been sold to ...
- 25
6
votes
1
answer
205
views
Bernoulli distribution with random means
Let $S = \frac{X_1 + \cdots + X_n}{n}$ where the $X_i$ are IID Bernoulli distributed with mean $p$, then $E[S] = p$ and $Var(S) = \frac{p(1-p)}{n}$.
Now consider the slightly more complex setup where $...
- 117
0
votes
1
answer
39
views
Properties of independent and identically distributed random variables [closed]
Do independent and identically distributed random variables always have the same expectation and variance?
- 1
6
votes
1
answer
111
views
References on data partitioning (cross-validation, train/val/test set construction) when data are non-IID
Consider a prediction setting in which we are interested in training a regression or classification function $f$ with inputs $X \in \mathbb{R}^k$ and target $Y$, and assessing its expected ...
- 3,768
8
votes
3
answers
665
views
Statistical learning when observations are not iid
As far as I am concerned, statistical/machine learning algorithms always suppose that data are independent and identically distributed ($iid$).
My question is: what can we do when this assumption is ...
- 352
1
vote
2
answers
53
views
Confusion about independent and identically distributed?
Say that I wish to measure the height of male within the population (so gender=Male is the only factor I am accounting for). Say I collect 100 observations of male height from an elderly population. ...
- 11
2
votes
2
answers
85
views
Asymptotic MLE Distribution With Two Random Samples
I'm studiyng for an exam, and I found this problem which I can not managed to solve... I will be really grateful if someone can help me, thanks you.
Let $\left\{X_{1}, \ldots, X_{n}\right\} \sim^{...
- 41
1
vote
0
answers
51
views
Central limit theorem for dependent binary-related variable
Let $Y\sim N(\mu, \sigma^2)$ and given sample size $n$, we have an iid sample $\{Y_1, ..., Y_n\}$. We sample $X$ (size $n$) from Bernoulli with probability $\pi$. Denote $Z_i=X_iY_i$. Then, when $X_i=...
- 789
1
vote
1
answer
132
views
Types of noise processes and the one assumed in arima() estimation in R
Here is a time series class defining white noise incorrectly as an independent sequence of random variables.
source
Aside from the widespread mix-up of White noise and iid noise, a further ...
- 3,242
3
votes
1
answer
645
views
What the i.i.d. assumption of the errors in linear regression implies for the response variable y?
In the linear regression model we assume that the errors $ε_i$ are independent and identically distributed (i.i.d.) random variables. I am trying to understand what this assumption implies regarding ...
- 671
0
votes
1
answer
71
views
AR/ARMA model - test residual for independence or lack of correlation
Shumway and Stoffer (2017), a great book that I highly recommend, define an AR(p) model as $x_t=\phi_1 x_{t-1}+\phi_2 x_{t-2}+...+\phi_p x_{t-p}+w_t$, where $w_t$~$wn(0,\sigma_w^2)$; $w_t$ is not ...
- 3,242
1
vote
0
answers
47
views
What is the meaning of these subscripts in iid random variables? [duplicate]
I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
0
votes
0
answers
21
views
How to calculate Maximum Likelihood Estimator
I have samples of a noisy real vector with constant phase y¯(2) = a¯ . e^jθ + w
where θ is a real scalar, and the entries of w¯ are complex normal i.i.d, where
W i,real, W i,image ∼ N(0,σ^2) for i = 1,...
- 121
0
votes
0
answers
81
views
MGF of sample mean of poisson distribution
Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$
Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$
Then, by Weak Law of Large Numbers ...
- 187
0
votes
1
answer
38
views
Clarifications on I.I.D. assumption in machine learning
In this question, it was stated that the assumption of i.i.d. for data comes in the form of
$$(X_i,y_i)∼P(X,y),∀i=1,...,N \\(X_i,y_i) \;independent\; of \;(X_j,y_j),\;∀i≠j∈{1,...,N}
$$
I am clear with ...
- 177
0
votes
1
answer
46
views
Confusion in definition of independent and identically distributed random variables [duplicate]
From what I learnt, a random variable is a function which assigns real values to outcome space, and the probability distribution is a function that assigns probability to different values produced by ...
0
votes
0
answers
50
views
Obtaining better estimates when you know that a set of input variables are independently and identically distributed
Suppose I have a family of random variables
$$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$
and I know how to sample $SomeDistribution_i$ for any $i$.
Suppose I also define a random variable $Y = f(...
- 379
2
votes
0
answers
50
views
ISI PCB NC$9$ Limiting Distribution of Bernoulli to Poisson
Let $X_i\sim (i.i.d.)$, Bernoulli($\frac{\lambda}{n}$), $n\ge \lambda\ge 0$.
$Y_i\sim (i.i.d.)$, Poisson($\frac{\lambda}{n}$). $\{X_i\}$ and $\{Y_i\}$ are independent.
Define $T_n=\sum_{i=1}^{n^2}X_i$...
- 121
3
votes
0
answers
43
views
Example of independent observations
I came across an interesting (to me, at least) question. I want to predict the probability that a given boat will win a race. The dataset is something like:
Y: winner (0-1)
X1: size of the sail
X2: ...
- 31
1
vote
0
answers
34
views
Trying to understand iid [closed]
Suppose that I have a sequence of random variables $\{X_i,Y_{it}\}_{i=1,t=1}^{N,T}$. I want to assume i.i.d with this sequence of random variables. So I wonder it is correct to assume
$\{X_i,Y_{it}\}...
- 637
3
votes
1
answer
64
views
Why is MAP and ML widely accepted? [closed]
(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori)
I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
- 43
0
votes
1
answer
118
views
Non IID data and SVM Classifier
I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
- 115
3
votes
1
answer
40
views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
- 516
2
votes
0
answers
217
views
Posterior distribution of two i.i.d. uniform r.v. given their difference with graphical intuition
I have two i.i.d. random variables, $\theta_1$ and $\theta_2$ which are uniformly distributed on the unit square. I need to compute the joint posterior distribution of these two variables, given their ...
- 31
0
votes
1
answer
56
views
Probability with two IID random variables
Assume $X$ and $Y$ are two IID random variables with infinite support, I am interested in $P(X>a , X>Y+b)$ where $a$ and $b$ are two constants. Is there any family of distribution for X and Y ...
- 83
3
votes
1
answer
84
views
Central limit theorem for the function of an iid random variable
Given an iid random variable $X$, instead of the distribution $\sqrt{n}(n^{-1}\sum{X_{i}}-E[X])$ which is the result that the central limit theorem provides , I am interested in the distribution of $\...
- 69
1
vote
0
answers
15
views
What happens if I use OLS in a multiple (hedonic) regression where prices are systematically missing?
I am using a hedonic regression of (log) housing prices on a set of price-determining characteristics. I then use the estimated coefficients to estimate housing prices for observations not in my ...
- 1,048
1
vote
1
answer
99
views
Definition of independent events in probability theory (Wasserman)
In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second ...
- 23
1
vote
0
answers
101
views
Are stationary markov chains iid random variables?
Let $\{X_t\}_{t=1}^{\infty}$ be a Markov Chain.
An initial marginal distribution $\pi^T$ for a markov chain is a stationary distribution if $\pi^TP = \pi^T$.
My understanding of this is that if the ...
- 23
1
vote
0
answers
59
views
Machine Learning IID [duplicate]
I am new in ML so excuse me if this is a bit basic.
I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent).
I don't exactly ...
- 33
0
votes
1
answer
172
views
i.i.d assumption: formal definition vs. intuition [duplicate]
Intuition
In ML, as I constantly run into the i.i.d assumption for datasets, I have an intuition of what this assumption really means. So if I'm not mistaken:
"independent" means that ...
- 125
0
votes
1
answer
61
views
Is there any need of random sampling for IID Data?
I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required ...
0
votes
1
answer
51
views
Iid random variables with infinite variance are unbounded
While preparing for an exam I've stumbled upon an exercise I have no idea how to approach:
$X_1, \dots, X_n$ are iid random variables with $E(X_1) = 0$ and $V(X_1)=\infty$
Show that $\mathbb{P}(\{ |...
- 3
2
votes
1
answer
84
views
Confounding variables VS i.i.d assumption
I made up an example so as to illustrate my question with some more context. Say there are two national parks, and a ranger is interested in finding out how the number of rabbits (Y) varies with the ...
- 23
3
votes
1
answer
108
views
What is the expectation of $\left\langle (n \bar{y})^4 \right\rangle$, if $y_i \sim \mathcal{N}(\mu,\sigma^2)$?
Let $y_i \sim \mathcal{N}(\mu,\sigma^2), \; i = 1,\ldots,n$ and $\bar{y} = \frac{1}{n} \sum_{i=1}^n y_i$, such that $n \bar{y} = y_1 + \ldots + y_n$.
Then, we want to know what the expectation of $(n \...
- 112
0
votes
0
answers
28
views
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? [duplicate]
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios?
My question is based off the following example from Wasserman's All of Statistics:
Suppose we test a prediction ...
1
vote
2
answers
945
views
Summation of i.i.d. Normal Random Variables
Assuming I have $$X_1,X_2,...,X_{100}\sim N(1,4)$$ and $$Y_1,Y_2,...,Y_{20}\sim N(2,9)$$ where all $X$ are iid, all $Y$ are iid.
Then should $$\text{var}(X_1+X_2+\ldots+X_{100}+Y_1+\ldots + Y_{20}) = ...
- 11
1
vote
1
answer
103
views
The non i.i.d. problem resulted from active learning query strategy
We usually assume the i.i.d. assumption in machine learning problems, but in active learning, the labeled examples acquired by querying oracle are clearly not i.i.d. I want to know will it be better ...
- 13
0
votes
1
answer
35
views
3
votes
0
answers
137
views
Finding a sub-population from dataset matching another target dataset
Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
- 136
0
votes
0
answers
140
views
Does permutation test require iid?
Suppose I have a treatment group X and independent from control group Y.
The data set x1,x2..., y1,y2... Let's say I want to test the mean of the difference
If the randomization unit is a session ...
- 167
6
votes
1
answer
131
views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
- 1,096
1
vote
2
answers
240
views
Independent and identically distributed data (images)?
If it is said that the data must be independent and identically distributed, and the data are images, then what exactly does it mean for images to be "independent and identically distributed"...
- 1,096
-1
votes
1
answer
168
views
Does a version of the Delta Method exist for non-i.i.d. sequences?
I have a sequence of random variables that are non-independent, but usually identically distributed. I am wondering if a version of the Delta Method exists under the case when I only have that the ...
- 2,609
1
vote
1
answer
197
views
Expected vallue calculation of i.i.d. random variables
Suppose $X_1,X_2,\ldots,X_n$ are a sequence of i.i.d. random variables with mean $\mu$ and variance $\sigma^2$.
Define the sample mean $\bar{X} := \frac{1}{n} \sum_{i=1}^{n} X_i$, which we know is an ...
- 125