Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
200
questions
0
votes
0answers
15 views
Is the expectation of a sample of independently and identically distributed random variables the expectation of just one of them> [closed]
What the title says. Since each 𝜉𝑖 is identically distributed, would it be correct to assume that the expectation and variance of one of them is the same as that of every other one? Would the ...
3
votes
1answer
56 views
Why is MAP and ML widely accepted? [closed]
(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori)
I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
0
votes
1answer
22 views
Non IID data and SVM Classifier
I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
3
votes
1answer
31 views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
2
votes
0answers
168 views
Posterior distribution of two i.i.d. uniform r.v. given their difference with graphical intuition
I have two i.i.d. random variables, $\theta_1$ and $\theta_2$ which are uniformly distributed on the unit square. I need to compute the joint posterior distribution of these two variables, given their ...
0
votes
1answer
33 views
Probability with two IID random variables
Assume $X$ and $Y$ are two IID random variables with infinite support, I am interested in $P(X>a , X>Y+b)$ where $a$ and $b$ are two constants. Is there any family of distribution for X and Y ...
3
votes
1answer
44 views
Central limit theorem for the function of an iid random variable
Given an iid random variable $X$, instead of the distribution $\sqrt{n}(n^{-1}\sum{X_{i}}-E[X])$ which is the result that the central limit theorem provides , I am interested in the distribution of $\...
1
vote
0answers
8 views
What happens if I use OLS in a multiple (hedonic) regression where prices are systematically missing?
I am using a hedonic regression of (log) housing prices on a set of price-determining characteristics. I then use the estimated coefficients to estimate housing prices for observations not in my ...
1
vote
1answer
35 views
Definition of independent events in probability theory (Wasserman)
In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second ...
0
votes
0answers
14 views
Example of two Random variables which are independent but from different distributions [duplicate]
I was reading about meaning of iid(independent but with identical distribution). Can there exist some 2 random variables which are independent but from different distributions? Any example?
0
votes
0answers
44 views
Are stationary markov chains iid random variables?
Let $\{X_t\}_{t=1}^{\infty}$ be a Markov Chain.
An initial marginal distribution $\pi^T$ for a markov chain is a stationary distribution if $\pi^TP = \pi^T$.
My understanding of this is that if the ...
1
vote
0answers
44 views
Machine Learning IID [duplicate]
I am new in ML so excuse me if this is a bit basic.
I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent).
I don't exactly ...
0
votes
1answer
64 views
i.i.d assumption: formal definition vs. intuition [duplicate]
Intuition
In ML, as I constantly run into the i.i.d assumption for datasets, I have an intuition of what this assumption really means. So if I'm not mistaken:
"independent" means that ...
0
votes
1answer
37 views
Is there any need of random sampling for IID Data?
I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required ...
0
votes
1answer
31 views
Iid random variables with infinite variance are unbounded
While preparing for an exam I've stumbled upon an exercise I have no idea how to approach:
$X_1, \dots, X_n$ are iid random variables with $E(X_1) = 0$ and $V(X_1)=\infty$
Show that $\mathbb{P}(\{ |...
2
votes
1answer
54 views
Confounding variables VS i.i.d assumption
I made up an example so as to illustrate my question with some more context. Say there are two national parks, and a ranger is interested in finding out how the number of rabbits (Y) varies with the ...
0
votes
0answers
32 views
What is the expectation of $\left\langle (n \bar{y})^4 \right\rangle$, if $y_i \sim \mathcal{N}(\mu,\sigma^2)$? [duplicate]
Let $y_i \sim \mathcal{N}(\mu,\sigma^2), \; i = 1,\ldots,n$ and $\bar{y} = \frac{1}{n} \sum_{i=1}^n y_i$, such that $n \bar{y} = y_1 + \ldots + y_n$.
Then, we want to know what the expectation of $(n \...
0
votes
0answers
16 views
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? [duplicate]
When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios?
My question is based off the following example from Wasserman's All of Statistics:
Suppose we test a prediction ...
1
vote
2answers
324 views
Summation of i.i.d. Normal Random Variables
Assuming I have $$X_1,X_2,...,X_{100}\sim N(1,4)$$ and $$Y_1,Y_2,...,Y_{20}\sim N(2,9)$$ where all $X$ are iid, all $Y$ are iid.
Then should $$\text{var}(X_1+X_2+\ldots+X_{100}+Y_1+\ldots + Y_{20}) = ...
1
vote
1answer
62 views
The non i.i.d. problem resulted from active learning query strategy
We usually assume the i.i.d. assumption in machine learning problems, but in active learning, the labeled examples acquired by querying oracle are clearly not i.i.d. I want to know will it be better ...
0
votes
1answer
32 views
3
votes
0answers
93 views
Finding a sub-population from dataset matching another target dataset
Let's say one has a finite collection of i.i.d. samples from an unknown source distribution $S=\{x_{i} | i \in [1,n_{S}], x_{i} \sim p_{X_{S}}(x)\}$. Where each $x$ is multidimensional and has ...
0
votes
0answers
11 views
Estimators that relate model accuracy to data distributions
Consider a set of dependent variables $D$, set of independent variables $I$ and $O_1, O_2,\ldots,O_N$ observations of these variables. The dataset is short and wide, $|D| = 3*N$.
For each $j \in I$, ...
0
votes
0answers
48 views
Does permutation test require iid?
Suppose I have a treatment group X and independent from control group Y.
The data set x1,x2..., y1,y2... Let's say I want to test the mean of the difference
If the randomization unit is a session ...
6
votes
1answer
70 views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
1
vote
2answers
87 views
Independent and identically distributed data (images)?
If it is said that the data must be independent and identically distributed, and the data are images, then what exactly does it mean for images to be "independent and identically distributed"...
-1
votes
1answer
114 views
Does a version of the Delta Method exist for non-i.i.d. sequences?
I have a sequence of random variables that are non-independent, but usually identically distributed. I am wondering if a version of the Delta Method exists under the case when I only have that the ...
1
vote
1answer
81 views
Expected vallue calculation of i.i.d. random variables
Suppose $X_1,X_2,\ldots,X_n$ are a sequence of i.i.d. random variables with mean $\mu$ and variance $\sigma^2$.
Define the sample mean $\bar{X} := \frac{1}{n} \sum_{i=1}^{n} X_i$, which we know is an ...
0
votes
0answers
40 views
How do you compare samples if they are not Independent and Identical Distributed (IID)?
I have 2 questions.
Nr. 1: I have a simulation model running for 1 year (after the warm-up period) that simulates the patients coming into a clinic for their treatment (set up daily) (a patient may ...
3
votes
2answers
181 views
What does i.i.d. mean for multivariate case?
When we say a random variable is i.i.d., it's often used to describe the dependency between the observations of that random variable, which I call the row dimension, indexed by time if it's a time ...
2
votes
1answer
48 views
Is there a central limit theorem for random variables with a bounded interval? [duplicate]
Is there any theorem which states the asymptotic distribution for the sample mean when the samples are drawn from a random variable which has a bounded interval?
1
vote
1answer
503 views
What is a random process with “stationary independent increments”?
I'm looking at a Solved Problem in "Schaum's Outline: Probability, Random Variables, and Random Processes", specifically Problem 5.21. In this problem it states:
Let $\{X(t), t \ge 0\}$ be ...
2
votes
1answer
107 views
How are cross validation and i.i.d. assumption of of a dataset related?
Is it necessary for the observations of the data set to be IID in order to use cross-validation on it? If so, why ? Could you explain in the context of a classification using decision tree.
0
votes
0answers
10 views
Apply panel data techniques to a panel where observation can be assumed to be independent?
I'm working with a data-set and am unsure if I should apply specific panel data techniques to it or not.
The data consists of panel data for municipalities in the Philippines and damage cause to rice ...
6
votes
1answer
201 views
Is a sample i.i.d or is a collection of random variables i.i.d.?
Basic terminology question.
I hear “let the sample be i.i.d.“ and “let these random variables be i.i.d.” being used interchangeably.
Even Wikipedia uses both:
A collection of random variables is ...
0
votes
0answers
13 views
Does the mean preserving spread of a distribution constitute a mean preserving spread of the joint distribution of two iid draws from it?
Let random vectors $X_1, X_2 \sim F, \;i.i.d, X_1, X_2 \in X $. Now replace $F$ with its mean-preserving spread (MPS), say $G$. My question is, does that constitute an MPS of the joint distribution of ...
3
votes
2answers
59 views
Accept-reject and subsets of iid samples
I have some confusion about subsets of iid samples being distributed as the original sample.
As an illustration, consider the accept-reject algorithm to produce iid samples from a pdf $f(x)$. We draw,...
1
vote
0answers
21 views
How to derive this MAE error bound on the central limit theorem?
Is this derived from Chebyshev's inequality or a tail bound theorem? If not, how was it derived?
Does this require the existence of the third moment?
Does this bound suggest the normal approximation ...
0
votes
0answers
57 views
I.I.D. for the layperson? [duplicate]
Question: For the layman, what does it mean for data (say $n$ samples covering $m$ variables) to be identically distributed, and how is it practically achieved when conducting machine learning?
So ...
1
vote
1answer
263 views
When using a gaussian link in GLM, what are the assumptions?
In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
0answers
228 views
How to compute ESS (Effective Sample Size)?
I implemented the ESS calculation according to this manual like this:
...
1
vote
0answers
44 views
What if my samples are not IIDs? [closed]
I'm testing my system and getting some results over time (Queues size, wating time, ecc.). When I plot the correlograms they exhibit a high degree of autocorrelation, even for high lags. Now, what are ...
0
votes
0answers
95 views
What exactly is $p(x,y)$ in the context of iid assumption in machine learning?
In machine learning iid assumption means that examples in the dataset are independent and drawn from the same probability distribution (i.e., identically distributed).
Here, the probability ...
14
votes
3answers
2k views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
2
votes
2answers
78 views
Clarifying the meaning of i.i.d. when describing a set of variables
Let $Z_1, ..., Z_k$ be identically and independently distributed (i.i.d.) set of standard normal random variables.
I understand that as part of the i.i.d. independent broadly means that variables ...
1
vote
0answers
24 views
Shouldn’t we say independent given the distribution?
In statistics we often deal with iid random variables: independent identically distributed. But if we don’t know the distribution (say we still know the support is {0, 1}), and we get a sample x1, say ...
2
votes
1answer
540 views
Why is the MLE for variance in single linear regression biased? [duplicate]
I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):
...
1
vote
0answers
618 views
Probability that one positive random variable is greater than another independent and identically distributed positive random variable
Suppose I have the following random variables $X_{0} \sim f(x)$ and $X_{1} \sim f(x)$ are independent. I want to know the probability that $X_{0} > X_{1}$. So I think I want to find:
$$ P(X_{0}>...
3
votes
1answer
767 views
Probability of a machine failing when components fail independently
Our machine is made out of 3634 units of component A which has failure rate of 10%, 1656 units of component B with failure rate of 35% and 3368 units of component C with failure rate of 55%. Assuming ...
3
votes
1answer
616 views
UCB Exploration in Reinforcement Learning
I have two questions regarding the upper confidence bounds (UCB) exploration in reinforcement learning:
UCB exploration is derived from Hoeffding's inequality which assumes that the reward is bounded ...
