Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
193
questions
3
votes
1answer
33 views
Central limit theorem for the function of an iid random variable
Given an iid random variable $X$, instead of the distribution $\sqrt{n}(n^{-1}\sum{X_{i}}-E[X])$ which is the result that the central limit theorem provides , I am interested in the distribution of $\...
1
vote
0answers
6 views
What happens if I use OLS in a multiple (hedonic) regression where prices are systematically missing?
I am using a hedonic regression of (log) housing prices on a set of price-determining characteristics. I then use the estimated coefficients to estimate housing prices for observations not in my ...
1
vote
1answer
27 views
Definition of independent events in probability theory (Wasserman)
In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second ...
0
votes
0answers
14 views
Example of two Random variables which are independent but from different distributions [duplicate]
I was reading about meaning of iid(independent but with identical distribution). Can there exist some 2 random variables which are independent but from different distributions? Any example?
0
votes
0answers
32 views
Are stationary markov chains iid random variables?
Let $\{X_t\}_{t=1}^{\infty}$ be a Markov Chain.
An initial marginal distribution $\pi^T$ for a markov chain is a stationary distribution if $\pi^TP = \pi^T$.
My understanding of this is that if the ...
1
vote
0answers
35 views
Machine Learning IID [duplicate]
I am new in ML so excuse me if this is a bit basic.
I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent).
I don't exactly ...
0
votes
1answer
45 views
i.i.d assumption: formal definition vs. intuition [duplicate]
Intuition
In ML, as I constantly run into the i.i.d assumption for datasets, I have an intuition of what this assumption really means. So if I'm not mistaken:
"independent" means that ...
0
votes
1answer
30 views
Is there any need of random sampling for IID Data?
I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required ...
0
votes
1answer
25 views
Iid random variables with infinite variance are unbounded
While preparing for an exam I've stumbled upon an exercise I have no idea how to approach:
$X_1, \dots, X_n$ are iid random variables with $E(X_1) = 0$ and $V(X_1)=\infty$
Show that $\mathbb{P}(\{ |...
2
votes
1answer
46 views
Confounding variables VS i.i.d assumption
I made up an example so as to illustrate my question with some more context. Say there are two national parks, and a ranger is interested in finding out how the number of rabbits (Y) varies with the ...
0
votes
0answers
22 views
What is the expectation of $\left\langle (n \bar{y})^4 \right\rangle$, if $y_i \sim \mathcal{N}(\mu,\sigma^2)$? [duplicate]
Let $y_i \sim \mathcal{N}(\mu,\sigma^2), \; i = 1,\ldots,n$ and $\bar{y} = \frac{1}{n} \sum_{i=1}^n y_i$, such that $n \bar{y} = y_1 + \ldots + y_n$.
Then, we want to know what the expectation of $(n \...
0
votes
0answers
11 views
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? [duplicate]
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios?
My question is based off the following example from Wasserman's All of Statistics:
Suppose we test a prediction ...
1
vote
2answers
164 views
Summation of i.i.d. Normal Random Variables
Assuming I have $$X_1,X_2,...,X_{100}\sim N(1,4)$$ and $$Y_1,Y_2,...,Y_{20}\sim N(2,9)$$ where all $X$ are iid, all $Y$ are iid.
Then should $$\text{var}(X_1+X_2+\ldots+X_{100}+Y_1+\ldots + Y_{20}) = ...
1
vote
1answer
41 views
The non i.i.d. problem resulted from active learning query strategy
We usually assume the i.i.d. assumption in machine learning problems, but in active learning, the labeled examples acquired by querying oracle are clearly not i.i.d. I want to know will it be better ...
0
votes
1answer
31 views
3
votes
0answers
72 views
Finding a sub-population from dataset matching another target dataset
Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
0
votes
0answers
11 views
Estimators that relate model accuracy to data distributions
Consider a set of dependent variables $D$, set of independent variables $I$ and $O_1, O_2,\ldots,O_N$ observations of these variables. The dataset is short and wide, $|D| = 3*N$.
For each $j \in I$, ...
0
votes
0answers
27 views
Does permutation test require iid?
Suppose I have a treatment group X and independent from control group Y.
The data set x1,x2..., y1,y2... Let's say I want to test the mean of the difference
If the randomization unit is a session ...
6
votes
1answer
59 views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
1
vote
2answers
62 views
Independent and identically distributed data (images)?
If it is said that the data must be independent and identically distributed, and the data are images, then what exactly does it mean for images to be "independent and identically distributed"...
-1
votes
1answer
91 views
Does a version of the Delta Method exist for non-i.i.d. sequences?
I have a sequence of random variables that are non-independent, but usually identically distributed. I am wondering if a version of the Delta Method exists under the case when I only have that the ...
1
vote
1answer
50 views
Expected vallue calculation of i.i.d. random variables
Suppose $X_1,X_2,\ldots,X_n$ are a sequence of i.i.d. random variables with mean $\mu$ and variance $\sigma^2$.
Define the sample mean $\bar{X} := \frac{1}{n} \sum_{i=1}^{n} X_i$, which we know is an ...
0
votes
0answers
39 views
How do you compare samples if they are not Independent and Identical Distributed (IID)?
I have 2 questions.
Nr. 1: I have a simulation model running for 1 year (after the warm-up period) that simulates the patients coming into a clinic for their treatment (set up daily) (a patient may ...
3
votes
2answers
118 views
What does i.i.d. mean for multivariate case?
When we say a random variable is i.i.d., it's often used to describe the dependency between the observations of that random variable, which I call the row dimension, indexed by time if it's a time ...
2
votes
1answer
44 views
Is there a central limit theorem for random variables with a bounded interval? [duplicate]
Is there any theorem which states the asymptotic distribution for the sample mean when the samples are drawn from a random variable which has a bounded interval?
1
vote
1answer
183 views
What is a random process with “stationary independent increments”?
I'm looking at a Solved Problem in "Schaum's Outline: Probability, Random Variables, and Random Processes", specifically Problem 5.21. In this problem it states:
Let $\{X(t), t \ge 0\}$ be ...
2
votes
1answer
67 views
How are cross validation and i.i.d. assumption of of a dataset related?
Is it necessary for the observations of the data set to be IID in order to use cross-validation on it? If so, why ? Could you explain in the context of a classification using decision tree.
0
votes
0answers
10 views
Apply panel data techniques to a panel where observation can be assumed to be independent?
I'm working with a data-set and am unsure if I should apply specific panel data techniques to it or not.
The data consists of panel data for municipalities in the Philippines and damage cause to rice ...
5
votes
1answer
145 views
Is a sample i.i.d or is a collection of random variables i.i.d.?
Basic terminology question.
I hear “let the sample be i.i.d.“ and “let these random variables be i.i.d.” being used interchangeably.
Even Wikipedia uses both:
A collection of random variables is ...
0
votes
0answers
12 views
Does the mean preserving spread of a distribution constitute a mean preserving spread of the joint distribution of two iid draws from it?
Let random vectors $X_1, X_2 \sim F, \;i.i.d, X_1, X_2 \in X $. Now replace $F$ with its mean-preserving spread (MPS), say $G$. My question is, does that constitute an MPS of the joint distribution of ...
3
votes
2answers
43 views
Accept-reject and subsets of iid samples
I have some confusion about subsets of iid samples being distributed as the original sample.
As an illustration, consider the accept-reject algorithm to produce iid samples from a pdf $f(x)$. We draw,...
1
vote
0answers
21 views
How to derive this MAE error bound on the central limit theorem?
Is this derived from Chebyshev's inequality or a tail bound theorem? If not, how was it derived?
Does this require the existence of the third moment?
Does this bound suggest the normal approximation ...
0
votes
0answers
50 views
I.I.D. for the layperson? [duplicate]
Question: For the layman, what does it mean for data (say $n$ samples covering $m$ variables) to be identically distributed, and how is it practically achieved when conducting machine learning?
So ...
0
votes
0answers
19 views
Is a non-iid process necessarily a Markov Process?
This may be a silly question, but I've been wondering if a a random variable is said to follow a non-iid process, does that necessarily imply that it follows a Markov process?
1
vote
1answer
187 views
When using a gaussian link in GLM, what are the assumptions?
In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
0answers
146 views
How to compute ESS (Effective Sample Size)?
I implemented the ESS calculation according to this manual like this:
...
1
vote
0answers
43 views
What if my samples are not IIDs? [closed]
I'm testing my system and getting some results over time (Queues size, wating time, ecc.). When I plot the correlograms they exhibit a high degree of autocorrelation, even for high lags. Now, what are ...
0
votes
0answers
83 views
What exactly is $p(x,y)$ in the context of iid assumption in machine learning?
In machine learning iid assumption means that examples in the dataset are independent and drawn from the same probability distribution (i.e., identically distributed).
Here, the probability ...
14
votes
3answers
2k views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
2
votes
2answers
71 views
Clarifying the meaning of i.i.d. when describing a set of variables
Let $Z_1, ..., Z_k$ be identically and independently distributed (i.i.d.) set of standard normal random variables.
I understand that as part of the i.i.d. independent broadly means that variables ...
1
vote
0answers
23 views
Shouldn’t we say independent given the distribution?
In statistics we often deal with iid random variables: independent identically distributed. But if we don’t know the distribution (say we still know the support is {0, 1}), and we get a sample x1, say ...
2
votes
1answer
358 views
Why is the MLE for variance in single linear regression biased? [duplicate]
I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):
...
1
vote
0answers
449 views
Probability that one positive random variable is greater than another independent and identically distributed positive random variable
Suppose I have the following random variables $X_{0} \sim f(x)$ and $X_{1} \sim f(x)$ are independent. I want to know the probability that $X_{0} > X_{1}$. So I think I want to find:
$$ P(X_{0}>...
3
votes
1answer
593 views
Probability of a machine failing when components fail independently
Our machine is made out of 3634 units of component A which has failure rate of 10%, 1656 units of component B with failure rate of 35% and 3368 units of component C with failure rate of 55%. Assuming ...
3
votes
1answer
522 views
UCB Exploration in Reinforcement Learning
I have two questions regarding the upper confidence bounds (UCB) exploration in reinforcement learning:
UCB exploration is derived from Hoeffding's inequality which assumes that the reward is bounded ...
0
votes
0answers
80 views
Bootstrapping regression coefficient for time series
One of the fundamental assumptions of bootstrap is that the samples are independent and identically distributed (i.i.d). This is the reason why it is difficult to bootstrap time-series because the ...
1
vote
2answers
632 views
Random Variable with IID always Gaussian?
Is there a case when we assume a random variable $\epsilon$ to be IID and assume its distribution is not gaussian?
0
votes
0answers
40 views
How to check Gauß-Markov theorem after OLS estimation?
I have estimate a simple bivariate regression with OLS and want to proof if the estimator is unbiased.
I found that the Gauß-Markov theorem consists of 4 Assumptions and the first 3 can be rewrite as ...
2
votes
1answer
184 views
Expectation Value of a Product of Many IID variables
First of all, I apologize for not being rigorous, but I am not a statistitian by background.
Imagine you have $N$ i.i.d. positive random variables $X_1...X_N$ and you are trying to compute a ...
3
votes
4answers
493 views
Check if residuals are IID (timeseries)
How can I check in R after decomposing a time series if my residual component is IID noise?
Would this be the best way (to use the autocorrelation function) and check for 0 correlation on all lags > ...
