Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
207
questions
3
votes
1answer
56 views
What the i.i.d. assumption of the errors in linear regression implies for the response variable y?
In the linear regression model we assume that the errors $ε_i$ are independent and identically distributed (i.i.d.) random variables. I am trying to understand what this assumption implies regarding ...
0
votes
1answer
20 views
AR/ARMA model - test residual for independence or lack of correlation
Shumway and Stoffer (2017), a great book that I highly recommend, define an AR(p) model as $x_t=\phi_1 x_{t-1}+\phi_2 x_{t-2}+...+\phi_p x_{t-p}+w_t$, where $w_t$~$wn(0,\sigma_w^2)$; $w_t$ is not ...
1
vote
0answers
43 views
What is the meaning of these subscripts in iid random variables? [duplicate]
I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
0
votes
0answers
16 views
How to calculate Maximum Likelihood Estimator
I have samples of a noisy real vector with constant phase y¯(2) = a¯ . e^jθ + w
where θ is a real scalar, and the entries of w¯ are complex normal i.i.d, where
W i,real, W i,image ∼ N(0,σ^2) for i = 1,...
0
votes
0answers
43 views
MGF of sample mean of poisson distribution
Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$
Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$
Then, by Weak Law of Large Numbers ...
0
votes
1answer
27 views
Clarifications on I.I.D. assumption in machine learning
In this question, it was stated that the assumption of i.i.d. for data comes in the form of
$$(X_i,y_i)∼P(X,y),∀i=1,...,N \\(X_i,y_i) \;independent\; of \;(X_j,y_j),\;∀i≠j∈{1,...,N}
$$
I am clear with ...
0
votes
1answer
31 views
Confusion in definition of independent and identically distributed random variables [duplicate]
From what I learnt, a random variable is a function which assigns real values to outcome space, and the probability distribution is a function that assigns probability to different values produced by ...
0
votes
0answers
49 views
Obtaining better estimates when you know that a set of input variables are independently and identically distributed
Suppose I have a family of random variables
$$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$
and I know how to sample $SomeDistribution_i$ for any $i$.
Suppose I also define a random variable $Y = f(...
2
votes
0answers
47 views
ISI PCB NC$9$ Limiting Distribution of Bernoulli to Poisson
Let $X_i\sim (i.i.d.)$, Bernoulli($\frac{\lambda}{n}$), $n\ge \lambda\ge 0$.
$Y_i\sim (i.i.d.)$, Poisson($\frac{\lambda}{n}$). $\{X_i\}$ and $\{Y_i\}$ are independent.
Define $T_n=\sum_{i=1}^{n^2}X_i$...
3
votes
0answers
18 views
Example of independent observations
I came across an interesting (to me, at least) question. I want to predict the probability that a given boat will win a race. The dataset is something like:
Y: winner (0-1)
X1: size of the sail
X2: ...
1
vote
0answers
34 views
Trying to understand iid [closed]
Suppose that I have a sequence of random variables $\{X_i,Y_{it}\}_{i=1,t=1}^{N,T}$. I want to assume i.i.d with this sequence of random variables. So I wonder it is correct to assume
$\{X_i,Y_{it}\}...
3
votes
1answer
61 views
Why is MAP and ML widely accepted? [closed]
(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori)
I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
0
votes
1answer
54 views
Non IID data and SVM Classifier
I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
3
votes
1answer
37 views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
2
votes
0answers
195 views
Posterior distribution of two i.i.d. uniform r.v. given their difference with graphical intuition
I have two i.i.d. random variables, $\theta_1$ and $\theta_2$ which are uniformly distributed on the unit square. I need to compute the joint posterior distribution of these two variables, given their ...
0
votes
1answer
39 views
Probability with two IID random variables
Assume $X$ and $Y$ are two IID random variables with infinite support, I am interested in $P(X>a , X>Y+b)$ where $a$ and $b$ are two constants. Is there any family of distribution for X and Y ...
3
votes
1answer
57 views
Central limit theorem for the function of an iid random variable
Given an iid random variable $X$, instead of the distribution $\sqrt{n}(n^{-1}\sum{X_{i}}-E[X])$ which is the result that the central limit theorem provides , I am interested in the distribution of $\...
1
vote
0answers
9 views
What happens if I use OLS in a multiple (hedonic) regression where prices are systematically missing?
I am using a hedonic regression of (log) housing prices on a set of price-determining characteristics. I then use the estimated coefficients to estimate housing prices for observations not in my ...
1
vote
1answer
66 views
Definition of independent events in probability theory (Wasserman)
In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second ...
0
votes
0answers
19 views
Example of two Random variables which are independent but from different distributions [duplicate]
I was reading about meaning of iid(independent but with identical distribution). Can there exist some 2 random variables which are independent but from different distributions? Any example?
1
vote
0answers
74 views
Are stationary markov chains iid random variables?
Let $\{X_t\}_{t=1}^{\infty}$ be a Markov Chain.
An initial marginal distribution $\pi^T$ for a markov chain is a stationary distribution if $\pi^TP = \pi^T$.
My understanding of this is that if the ...
1
vote
0answers
54 views
Machine Learning IID [duplicate]
I am new in ML so excuse me if this is a bit basic.
I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent).
I don't exactly ...
0
votes
1answer
108 views
i.i.d assumption: formal definition vs. intuition [duplicate]
Intuition
In ML, as I constantly run into the i.i.d assumption for datasets, I have an intuition of what this assumption really means. So if I'm not mistaken:
"independent" means that ...
0
votes
1answer
48 views
Is there any need of random sampling for IID Data?
I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required ...
0
votes
1answer
37 views
Iid random variables with infinite variance are unbounded
While preparing for an exam I've stumbled upon an exercise I have no idea how to approach:
$X_1, \dots, X_n$ are iid random variables with $E(X_1) = 0$ and $V(X_1)=\infty$
Show that $\mathbb{P}(\{ |...
2
votes
1answer
68 views
Confounding variables VS i.i.d assumption
I made up an example so as to illustrate my question with some more context. Say there are two national parks, and a ranger is interested in finding out how the number of rabbits (Y) varies with the ...
3
votes
1answer
90 views
What is the expectation of $\left\langle (n \bar{y})^4 \right\rangle$, if $y_i \sim \mathcal{N}(\mu,\sigma^2)$?
Let $y_i \sim \mathcal{N}(\mu,\sigma^2), \; i = 1,\ldots,n$ and $\bar{y} = \frac{1}{n} \sum_{i=1}^n y_i$, such that $n \bar{y} = y_1 + \ldots + y_n$.
Then, we want to know what the expectation of $(n \...
0
votes
0answers
21 views
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? [duplicate]
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios?
My question is based off the following example from Wasserman's All of Statistics:
Suppose we test a prediction ...
1
vote
2answers
579 views
Summation of i.i.d. Normal Random Variables
Assuming I have $$X_1,X_2,...,X_{100}\sim N(1,4)$$ and $$Y_1,Y_2,...,Y_{20}\sim N(2,9)$$ where all $X$ are iid, all $Y$ are iid.
Then should $$\text{var}(X_1+X_2+\ldots+X_{100}+Y_1+\ldots + Y_{20}) = ...
1
vote
1answer
79 views
The non i.i.d. problem resulted from active learning query strategy
We usually assume the i.i.d. assumption in machine learning problems, but in active learning, the labeled examples acquired by querying oracle are clearly not i.i.d. I want to know will it be better ...
0
votes
1answer
33 views
3
votes
0answers
128 views
Finding a sub-population from dataset matching another target dataset
Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
0
votes
0answers
96 views
Does permutation test require iid?
Suppose I have a treatment group X and independent from control group Y.
The data set x1,x2..., y1,y2... Let's say I want to test the mean of the difference
If the randomization unit is a session ...
6
votes
1answer
106 views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
1
vote
2answers
140 views
Independent and identically distributed data (images)?
If it is said that the data must be independent and identically distributed, and the data are images, then what exactly does it mean for images to be "independent and identically distributed"...
-1
votes
1answer
144 views
Does a version of the Delta Method exist for non-i.i.d. sequences?
I have a sequence of random variables that are non-independent, but usually identically distributed. I am wondering if a version of the Delta Method exists under the case when I only have that the ...
1
vote
1answer
136 views
Expected vallue calculation of i.i.d. random variables
Suppose $X_1,X_2,\ldots,X_n$ are a sequence of i.i.d. random variables with mean $\mu$ and variance $\sigma^2$.
Define the sample mean $\bar{X} := \frac{1}{n} \sum_{i=1}^{n} X_i$, which we know is an ...
3
votes
2answers
280 views
What does i.i.d. mean for multivariate case?
When we say a random variable is i.i.d., it's often used to describe the dependency between the observations of that random variable, which I call the row dimension, indexed by time if it's a time ...
2
votes
1answer
60 views
Is there a central limit theorem for random variables with a bounded interval? [duplicate]
Is there any theorem which states the asymptotic distribution for the sample mean when the samples are drawn from a random variable which has a bounded interval?
1
vote
1answer
936 views
What is a random process with "stationary independent increments"?
I'm looking at a Solved Problem in "Schaum's Outline: Probability, Random Variables, and Random Processes", specifically Problem 5.21. In this problem it states:
Let $\{X(t), t \ge 0\}$ be ...
3
votes
1answer
189 views
How are cross validation and i.i.d. assumption of of a dataset related?
Is it necessary for the observations of the data set to be IID in order to use cross-validation on it? If so, why ? Could you explain in the context of a classification using decision tree.
6
votes
1answer
253 views
Is a sample i.i.d or is a collection of random variables i.i.d.?
Basic terminology question.
I hear “let the sample be i.i.d.“ and “let these random variables be i.i.d.” being used interchangeably.
Even Wikipedia uses both:
A collection of random variables is ...
3
votes
2answers
67 views
Accept-reject and subsets of iid samples
I have some confusion about subsets of iid samples being distributed as the original sample.
As an illustration, consider the accept-reject algorithm to produce iid samples from a pdf $f(x)$. We draw,...
1
vote
0answers
24 views
How to derive this MAE error bound on the central limit theorem?
Is this derived from Chebyshev's inequality or a tail bound theorem? If not, how was it derived?
Does this require the existence of the third moment?
Does this bound suggest the normal approximation ...
0
votes
0answers
64 views
I.I.D. for the layperson? [duplicate]
Question: For the layman, what does it mean for data (say $n$ samples covering $m$ variables) to be identically distributed, and how is it practically achieved when conducting machine learning?
So ...
1
vote
1answer
375 views
When using a gaussian link in GLM, what are the assumptions?
In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
1answer
310 views
How to compute ESS (Effective Sample Size)?
I implemented the ESS calculation according to this manual like this:
...
1
vote
0answers
47 views
What if my samples are not IIDs? [closed]
I'm testing my system and getting some results over time (Queues size, wating time, ecc.). When I plot the correlograms they exhibit a high degree of autocorrelation, even for high lags. Now, what are ...
0
votes
0answers
140 views
What exactly is $p(x,y)$ in the context of iid assumption in machine learning?
In machine learning iid assumption means that examples in the dataset are independent and drawn from the same probability distribution (i.e., identically distributed).
Here, the probability ...
15
votes
3answers
2k views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
