Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
228
questions
3
votes
0
answers
21
views
Are there examples of ML or stats approaches that are valid for IID data, but not exchangeable data?
Lots of supervised learning theory is motivated using the IID assumption. Do most of these methods apply equally well if data is only exchangeable, and not IID? Can you provide an example where this ...
- 316
0
votes
0
answers
17
views
What panel data assumptions hold true when the underlying process is pooled cross sectional?
Most econometric textbooks convey that panel data contains both an entity and a time component, and that the entity can be individuals, firms, counties etc. The assumption we need to deal with in ...
- 1
0
votes
0
answers
55
views
What is the CDF of sum of 2 Pareto random variables?
Assume we have two independent, random variables, $X$ and $Y$ which follow the same Pareto distribution, i.e.:
$f(x) = \frac{1}{x^{2}}, x > 1$
Given that - what would be the joint probability $P(X +...
- 13
1
vote
1
answer
118
views
Central limit theorem for asymptotically i.i.d. random variables
I observe a sequence of r.v. $X_1, X_2, \dots$ where each $X_i$ is a function of the sample size $n$.
When $n \rightarrow \infty$ I have the following result: $X_1 \rightarrow^d E_1, X_2 \rightarrow^d ...
- 63
0
votes
0
answers
28
views
Steiger's Z test, independent samples required?
I have some samples and for each one, I have 3 measures : x, y and z. I want to know if the correlation between x and y and the correlation between y and z are significantly different. I will use ...
2
votes
2
answers
67
views
If $X_1,\ldots,X_n$ are continuous and IID with sample median $m$, are $1(X_1\geq m),\ldots,1(X_n\geq m)$ IID?
Suppose $X_1,\ldots,X_n$ are continuous random variables and IID with sample median $m$ for some $n\geq 2$. Is it then true that $1(X_1\geq m),\ldots,1(X_n\geq m)$ are IID?
Here the definition of $1(...
- 1,068
1
vote
0
answers
24
views
Data collection after the model is built and deployed
I have built a machine learning model which predicts whether a customer will buy a product or not. The model performs well on cross validation tests. Now, I will deploy it in production to recommend ...
- 1,023
2
votes
1
answer
38
views
Does the assumption of “independent and identically distributed” apply to the actual sample data or the sampling process?
While studying statistics, I came across the concept of “independent and identically distributed random variables” or IID.
I’m confused as to what that applies to in practice, mainly, does it apply to ...
- 23
0
votes
0
answers
25
views
$E(SN)$ for aggregate claim amount $S$, $S=X_{1}+...+X_{n}, X_{i}$ are iid [duplicate]
Consider the following model for aggregate claim amounts $S$:
$S=X_{1}+X_{2}+...+X_{N}$
where the $X_{i}$ are independent, identically distributed random variables representing individual claim ...
1
vote
1
answer
61
views
Studying extreme value r.v. $X=\max_i (c_i+X_i)$ where $c_i$ are constants and $X_i$ are i.i.d. r.v
Let
$X_1,X_2,...,X_n$ be independently and identically distributed random variables according to a distribution $F$.
There are constants: $c_1,c_2,...,c_n$.
Define a new random variable $X=\max_i(X_i+...
- 143
2
votes
0
answers
44
views
Test for distribution equality
This question touches Kolmogorov-Smirnov testing, but asks actually something different.
Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis:
$$
H_0: X_i\ ...
- 145
2
votes
1
answer
121
views
MCMC Sample should be i.i.d
I'm a bit not sure how to show that MCMC samples are i.i.d. In my opinion the trace plot should behave like white noise model because white noise model has a strong stationary properties i.e. the ...
0
votes
0
answers
168
views
If we remove half the samples from an IID dataset, is the remaining half still IID?
I am generating 10,000 pairs of X and Y such that both X and ...
- 241
0
votes
0
answers
37
views
understanding "independent" term in "independent and identically distributed" (iid) statement
I understand independent as the random variable values in a data aren't connected to each other in any way.
(q1) Is my understanding ok?
data_1: a product which has a design defect has been sold to ...
- 25
6
votes
1
answer
219
views
Bernoulli distribution with random means
Let $S = \frac{X_1 + \cdots + X_n}{n}$ where the $X_i$ are IID Bernoulli distributed with mean $p$, then $E[S] = p$ and $Var(S) = \frac{p(1-p)}{n}$.
Now consider the slightly more complex setup where $...
- 127
0
votes
1
answer
131
views
Properties of independent and identically distributed random variables [closed]
Do independent and identically distributed random variables always have the same expectation and variance?
- 1
6
votes
1
answer
174
views
References on data partitioning (cross-validation, train/val/test set construction) when data are non-IID
Consider a prediction setting in which we are interested in training a regression or classification function $f$ with inputs $X \in \mathbb{R}^k$ and target $Y$, and assessing its expected ...
- 3,850
9
votes
3
answers
825
views
Statistical learning when observations are not iid
As far as I am concerned, statistical/machine learning algorithms always suppose that data are independent and identically distributed ($iid$).
My question is: what can we do when this assumption is ...
- 361
1
vote
2
answers
80
views
Confusion about independent and identically distributed?
Say that I wish to measure the height of male within the population (so gender=Male is the only factor I am accounting for). Say I collect 100 observations of male height from an elderly population. ...
- 11
2
votes
2
answers
104
views
Asymptotic MLE Distribution With Two Random Samples
I'm studiyng for an exam, and I found this problem which I can not managed to solve... I will be really grateful if someone can help me, thanks you.
Let $\left\{X_{1}, \ldots, X_{n}\right\} \sim^{...
- 41
1
vote
0
answers
66
views
Central limit theorem for dependent binary-related variable
Let $Y\sim N(\mu, \sigma^2)$ and given sample size $n$, we have an iid sample $\{Y_1, ..., Y_n\}$. We sample $X$ (size $n$) from Bernoulli with probability $\pi$. Denote $Z_i=X_iY_i$. Then, when $X_i=...
- 799
1
vote
1
answer
197
views
Types of noise processes and the one assumed in arima() estimation in R
Here is a time series class defining white noise incorrectly as an independent sequence of random variables.
source
Aside from the widespread mix-up of White noise and iid noise, a further ...
- 3,490
3
votes
1
answer
2k
views
What the i.i.d. assumption of the errors in linear regression implies for the response variable y?
In the linear regression model we assume that the errors $ε_i$ are independent and identically distributed (i.i.d.) random variables. I am trying to understand what this assumption implies regarding ...
- 691
0
votes
1
answer
127
views
AR/ARMA model - test residual for independence or lack of correlation
Shumway and Stoffer (2017), a great book that I highly recommend, define an AR(p) model as $x_t=\phi_1 x_{t-1}+\phi_2 x_{t-2}+...+\phi_p x_{t-p}+w_t$, where $w_t$~$wn(0,\sigma_w^2)$; $w_t$ is not ...
- 3,490
1
vote
0
answers
47
views
What is the meaning of these subscripts in iid random variables? [duplicate]
I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
0
votes
0
answers
22
views
How to calculate Maximum Likelihood Estimator
I have samples of a noisy real vector with constant phase y¯(2) = a¯ . e^jθ + w
where θ is a real scalar, and the entries of w¯ are complex normal i.i.d, where
W i,real, W i,image ∼ N(0,σ^2) for i = 1,...
- 121
0
votes
0
answers
96
views
MGF of sample mean of poisson distribution
Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$
Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$
Then, by Weak Law of Large Numbers ...
- 187
0
votes
1
answer
63
views
Clarifications on I.I.D. assumption in machine learning
In this question, it was stated that the assumption of i.i.d. for data comes in the form of
$$(X_i,y_i)∼P(X,y),∀i=1,...,N \\(X_i,y_i) \;independent\; of \;(X_j,y_j),\;∀i≠j∈{1,...,N}
$$
I am clear with ...
- 177
0
votes
1
answer
50
views
Confusion in definition of independent and identically distributed random variables [duplicate]
From what I learnt, a random variable is a function which assigns real values to outcome space, and the probability distribution is a function that assigns probability to different values produced by ...
0
votes
0
answers
54
views
Obtaining better estimates when you know that a set of input variables are independently and identically distributed
Suppose I have a family of random variables
$$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$
and I know how to sample $SomeDistribution_i$ for any $i$.
Suppose I also define a random variable $Y = f(...
- 389
2
votes
0
answers
50
views
ISI PCB NC$9$ Limiting Distribution of Bernoulli to Poisson
Let $X_i\sim (i.i.d.)$, Bernoulli($\frac{\lambda}{n}$), $n\ge \lambda\ge 0$.
$Y_i\sim (i.i.d.)$, Poisson($\frac{\lambda}{n}$). $\{X_i\}$ and $\{Y_i\}$ are independent.
Define $T_n=\sum_{i=1}^{n^2}X_i$...
- 121
3
votes
0
answers
84
views
Example of independent observations
I came across an interesting (to me, at least) question. I want to predict the probability that a given boat will win a race. The dataset is something like:
Y: winner (0-1)
X1: size of the sail
X2: ...
- 31
1
vote
0
answers
34
views
Trying to understand iid [closed]
Suppose that I have a sequence of random variables $\{X_i,Y_{it}\}_{i=1,t=1}^{N,T}$. I want to assume i.i.d with this sequence of random variables. So I wonder it is correct to assume
$\{X_i,Y_{it}\}...
- 637
3
votes
1
answer
71
views
Why is MAP and ML widely accepted? [closed]
(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori)
I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
- 43
0
votes
1
answer
189
views
Non IID data and SVM Classifier
I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
- 115
3
votes
1
answer
45
views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
- 526
2
votes
0
answers
226
views
Posterior distribution of two i.i.d. uniform r.v. given their difference with graphical intuition
I have two i.i.d. random variables, $\theta_1$ and $\theta_2$ which are uniformly distributed on the unit square. I need to compute the joint posterior distribution of these two variables, given their ...
- 31
0
votes
1
answer
71
views
Probability with two IID random variables
Assume $X$ and $Y$ are two IID random variables with infinite support, I am interested in $P(X>a , X>Y+b)$ where $a$ and $b$ are two constants. Is there any family of distribution for X and Y ...
- 83
3
votes
1
answer
106
views
Central limit theorem for the function of an iid random variable
Given an iid random variable $X$, instead of the distribution $\sqrt{n}(n^{-1}\sum{X_{i}}-E[X])$ which is the result that the central limit theorem provides , I am interested in the distribution of $\...
- 134
1
vote
0
answers
20
views
What happens if I use OLS in a multiple (hedonic) regression where prices are systematically missing?
I am using a hedonic regression of (log) housing prices on a set of price-determining characteristics. I then use the estimated coefficients to estimate housing prices for observations not in my ...
- 1,168
1
vote
1
answer
137
views
Definition of independent events in probability theory (Wasserman)
In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second ...
- 23
1
vote
0
answers
114
views
Are stationary markov chains iid random variables?
Let $\{X_t\}_{t=1}^{\infty}$ be a Markov Chain.
An initial marginal distribution $\pi^T$ for a markov chain is a stationary distribution if $\pi^TP = \pi^T$.
My understanding of this is that if the ...
- 33
1
vote
0
answers
72
views
Machine Learning IID [duplicate]
I am new in ML so excuse me if this is a bit basic.
I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent).
I don't exactly ...
- 43
1
vote
1
answer
238
views
i.i.d assumption: formal definition vs. intuition [duplicate]
Intuition
In ML, as I constantly run into the i.i.d assumption for datasets, I have an intuition of what this assumption really means. So if I'm not mistaken:
"independent" means that ...
- 135
0
votes
1
answer
64
views
Is there any need of random sampling for IID Data?
I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required ...
0
votes
1
answer
60
views
Iid random variables with infinite variance are unbounded
While preparing for an exam I've stumbled upon an exercise I have no idea how to approach:
$X_1, \dots, X_n$ are iid random variables with $E(X_1) = 0$ and $V(X_1)=\infty$
Show that $\mathbb{P}(\{ |...
- 3
2
votes
1
answer
100
views
Confounding variables VS i.i.d assumption
I made up an example so as to illustrate my question with some more context. Say there are two national parks, and a ranger is interested in finding out how the number of rabbits (Y) varies with the ...
- 23
3
votes
1
answer
113
views
What is the expectation of $\left\langle (n \bar{y})^4 \right\rangle$, if $y_i \sim \mathcal{N}(\mu,\sigma^2)$?
Let $y_i \sim \mathcal{N}(\mu,\sigma^2), \; i = 1,\ldots,n$ and $\bar{y} = \frac{1}{n} \sum_{i=1}^n y_i$, such that $n \bar{y} = y_1 + \ldots + y_n$.
Then, we want to know what the expectation of $(n \...
- 112
0
votes
0
answers
32
views
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? [duplicate]
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios?
My question is based off the following example from Wasserman's All of Statistics:
Suppose we test a prediction ...
1
vote
2
answers
1k
views
Summation of i.i.d. Normal Random Variables
Assuming I have $$X_1,X_2,...,X_{100}\sim N(1,4)$$ and $$Y_1,Y_2,...,Y_{20}\sim N(2,9)$$ where all $X$ are iid, all $Y$ are iid.
Then should $$\text{var}(X_1+X_2+\ldots+X_{100}+Y_1+\ldots + Y_{20}) = ...
- 11