Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
164
questions
1
vote
1answer
35 views
How are cross validation and i.i.d. assumption of of a dataset related?
Is it necessary for the observations of the data set to be IID in order to use cross-validation on it? If so, why ? Could you explain in the context of a classification using decision tree.
0
votes
0answers
9 views
Apply panel data techniques to a panel where observation can be assumed to be independent?
I'm working with a data-set and am unsure if I should apply specific panel data techniques to it or not.
The data consists of panel data for municipalities in the Philippines and damage cause to rice ...
4
votes
1answer
67 views
Is a sample i.i.d or is a collection of random variables i.i.d.?
Basic terminology question.
I hear “let the sample be i.i.d.“ and “let these random variables be i.i.d.” being used interchangeably.
Even Wikipedia uses both:
A collection of random variables is ...
0
votes
0answers
8 views
Does the mean preserving spread of a distribution constitute a mean preserving spread of the joint distribution of two iid draws from it?
Let random vectors $X_1, X_2 \sim F, \;i.i.d, X_1, X_2 \in X $. Now replace $F$ with its mean-preserving spread (MPS), say $G$. My question is, does that constitute an MPS of the joint distribution of ...
2
votes
2answers
28 views
Accept-reject and subsets of iid samples
I have some confusion about subsets of iid samples being distributed as the original sample.
As an illustration, consider the accept-reject algorithm to produce iid samples from a pdf $f(x)$. We draw,...
1
vote
0answers
16 views
How to derive this MAE error bound on the central limit theorem?
Is this derived from Chebyshev's inequality or a tail bound theorem? If not, how was it derived?
Does this require the existence of the third moment?
Does this bound suggest the normal approximation ...
0
votes
0answers
39 views
I.I.D. for the layperson? [duplicate]
Question: For the layman, what does it mean for data (say $n$ samples covering $m$ variables) to be identically distributed, and how is it practically achieved when conducting machine learning?
So ...
0
votes
0answers
14 views
Is a non-iid process necessarily a Markov Process?
This may be a silly question, but I've been wondering if a a random variable is said to follow a non-iid process, does that necessarily imply that it follows a Markov process?
0
votes
1answer
69 views
When using a gaussian link in GLM, what are the assumptions?
In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
0answers
20 views
How to compute ESS (Effective Sample Size)?
I implemented the ESS calculation according to this manual like this:
...
1
vote
0answers
43 views
What if my samples are not IIDs? [closed]
I'm testing my system and getting some results over time (Queues size, wating time, ecc.). When I plot the correlograms they exhibit a high degree of autocorrelation, even for high lags. Now, what are ...
0
votes
0answers
53 views
What exactly is p(x,y) in the context of iid assumption in machine learning?
In machine learning iid assumption means that examples in the dataset are independent and drawn from the same probability distribution (i.e., identically distributed).
Here, the probability ...
12
votes
3answers
1k views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
1
vote
0answers
39 views
What problems happen when $L(\theta|x_1,x_2,…,x_n)$ be a function of differently distributed random variables?
Given $L(\theta|x_1,x_2,...,x_n)$, may $x_1,x_2,...,x_n$ not all be identically distributed? What problems happen in the not identically distributed case compared to the iid case?
1
vote
0answers
21 views
Shouldn’t we say independent given the distribution?
In statistics we often deal with iid random variables: independent identically distributed. But if we don’t know the distribution (say we still know the support is {0, 1}), and we get a sample x1, say ...
2
votes
1answer
129 views
Why is the MLE for variance in single linear regression biased? [duplicate]
I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):
...
1
vote
0answers
159 views
Probability that one positive random variable is greater than another independent and identically distributed positive random variable
Suppose I have the following random variables $X_{0} \sim f(x)$ and $X_{1} \sim f(x)$ are independent. I want to know the probability that $X_{0} > X_{1}$. So I think I want to find:
$$ P(X_{0}>...
2
votes
1answer
193 views
Probability of a machine failing when components fail independently
Our machine is made out of 3634 units of component A which has failure rate of 10%, 1656 units of component B with failure rate of 35% and 3368 units of component C with failure rate of 55%. Assuming ...
2
votes
1answer
339 views
UCB Exploration in Reinforcement Learning
I have two questions regarding the upper confidence bounds (UCB) exploration in reinforcement learning:
UCB exploration is derived from Hoeffding's inequality which assumes that the reward is bounded ...
0
votes
0answers
52 views
Bootstrapping regression coefficient for time series
One of the fundamental assumptions of bootstrap is that the samples are independent and identically distributed (i.i.d). This is the reason why it is difficult to bootstrap time-series because the ...
1
vote
2answers
238 views
Random Variable with IID always Gaussian?
Is there a case when we assume a random variable $\epsilon$ to be IID and assume its distribution is not gaussian?
0
votes
0answers
32 views
How to check Gauß-Markov theorem after OLS estimation?
I have estimate a simple bivariate regression with OLS and want to proof if the estimator is unbiased.
I found that the Gauß-Markov theorem consists of 4 Assumptions and the first 3 can be rewrite as ...
2
votes
1answer
96 views
Expectation Value of a Product of Many IID variables
First of all, I apologize for not being rigorous, but I am not a statistitian by background.
Imagine you have $N$ i.i.d. positive random variables $X_1...X_N$ and you are trying to compute a ...
1
vote
3answers
205 views
Check if residuals are IID (timeseries)
How can I check in R after decomposing a time series if my residual component is IID noise?
Would this be the best way (to use the autocorrelation function) and check for 0 correlation on all lags > ...
4
votes
0answers
197 views
What is the fundamental assumption for using resampling methods?
Suppose that I observe a set of non-i.i.d. data (time series) $\mathcal{L} = \left\lbrace (y_{t}, x_{t}) \right\rbrace_{t=1}^{T}$ with $x_{t} = (x_{t1},\ldots,x_{tP})$ a real valued vector of $P$ ...
1
vote
0answers
28 views
When does $E[f(X_i)]=E[f(X_j)], i\neq j$?
Suppose we have random variables $X_1, \dots, X_N$, with joint probability distribution $F_{X_1,\dots,X_N}$. Under what conditions does the following equality holds?
$$E[f(X_i)]=E[f(X_j)],\ \ i\neq ...
6
votes
2answers
392 views
Expectation of inverse of sum of positive iid variables
Let $(X_i)_{i}$ be a sequence of iid positive variables of mean 1 and variance $\sigma^2$. Let $\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$.
My question is: Can we can bound $\mathbb{E}(1/\bar{X}_n)$ as ...
0
votes
1answer
258 views
Are i.i.d. random variables always associated with a distribution function? [closed]
It is known that independent and identically distributed (i.i.d.) random variables are mutually independent and each random variable has the same probability distribution as the others.
However, is ...
1
vote
1answer
63 views
Implications of i.i.d. sample
I have the following question:
I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as
$\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
46 views
Are feature values random variable realizations, or random variables themselves?
I'm new to ML/stats so got confused with what I supposed was simple notation.
For simplicity, say I have a data set with just one column:
From probabilistic perspective, I had understood that X ...
6
votes
1answer
82 views
Is MLE intrinsically connected to logs?
My mathematical exploration led me the following claim:
Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
1
vote
0answers
36 views
Functional data analysis references
I am working on a inferential framework for functional data. I have categorical independent variables and a functional response variables. Different implementations of functional linear models are ...
3
votes
1answer
188 views
Does joint IID imply marginal IID?
Suppose you have a 2 dimension random vector denoted $(X,Y)$ that is Independent and Identically Distributed (IID) for a sequence of draws $((X_1,Y_1),(X_2,Y_2),...,(X_n,Y_n))$. Does this imply that ...
7
votes
3answers
162 views
Calculate the variance of $\sum\limits_{i=1}^{n-1} \sum\limits_{j=i+1}^n S(X_i - X_j)$ for $X_1,\ldots,X_n$ i.i.d. random variables
In p.88 of Wand & Jones (1995), they asked to show the following result.
Let $X_1,\ldots,X_n$ be a set of i.i.d. random variables and define $$U=2n^{-2}\sum_{i=1}^{n-1} \sum_{j=i+1}^n S(X_i - ...
5
votes
2answers
236 views
Identically distributed vs P(X > Y) = P(Y > X)
I've two related propositions which seem correct intuitively, but I struggle to prove them properly.
Question 1
Prove or disprove: If $X$ and $Y$ are independent and have identical marginal ...
1
vote
0answers
25 views
What final test to apply to a permutation test (by compression) algorithm?
I'm following in the steps of §5.1 Permutation Testing, NIST Special Publication 800-90B, Recommendation for the Entropy Sources Used for Random Bit Generation.
I'm developing my test for ...
0
votes
1answer
89 views
Likelihood when points aren't i.i.d
If we assume that we have a set of N data points given as $\textbf{X}$ and corresponding targets vectors $\textbf{T}$, where both represents matrices in this case. For an i.i.d we could write the ...
1
vote
0answers
107 views
Estimate unknown sum of iid random variables
Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known.
...
3
votes
1answer
185 views
Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID
From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
0answers
168 views
Expectation of product
Let $\{X_i\}_{i\in I}$ be a finite collection of i.i.d random variables. I have found that $$E[\prod_i X_i]=\prod_i E[X_i]$$ But I haven't found a proof of this fact. What is the proof?
1
vote
0answers
30 views
Minimum of i.i.d. Random Variables [duplicate]
What importance does the minimum of an identical and independently distributed random variables play in probability distribution?
1
vote
1answer
134 views
Role of random sample assumption in consistency of OLS estimator
I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it?
I am confused because for example I have ...
4
votes
1answer
58 views
A symmetric iid process
Let $X_1, X_2, \ldots$ be an iid process with $X_i$ having a symmetric distribution around $0$. Then can I always write
$$X_1 - \alpha X_{t-1}-\alpha^2 X_{t-2}-\cdots \stackrel{iid}{=} X_1 + |\alpha| ...
1
vote
1answer
145 views
Covariance between a variable and a non-linear transformation of it
Suppose $\epsilon \overset{\text{iid}}{\sim} N(0, \sigma^2)$
Can we make any assumptions about Cov$(\epsilon, \frac{\epsilon^2}{1 + \epsilon^2})$?
1
vote
0answers
66 views
Can i say an iid process with $0$ mean has homogeneous independent increment?
I think the title is itself self-explanatory. My idea says that an IID process with $0$ mean has to have independent increment. But, I do not understand how to prove homogeneity?
4
votes
2answers
79 views
Distribution of EMA of iid $\chi_1^2$
I am wondering if it is possible to get an analytical expression for the distribution of a random variable defined as
$$
r = \sum_{k=0}^\infty a_i b^i
$$
where $a_i$ are iid chi-square variable ...
0
votes
0answers
30 views
Wind (speed) - a non-iid random variable?
Over a time period of 1 year, half hourly wind speed data is collected. Is it false to state that the collected data is not-iid? I know that wind is seen as a non-stationary process, but does it also ...
1
vote
1answer
30 views
Samples drawn(iid) from univariate gaussian. Does their combination is drawn from multivariate gaussian
If $x_i \sim \mathrm{N}(0,1)$ for i = 1:n and $x_i$ are iid, is it true that $(x_1, x_2, ...,x_{n}) \sim \mathrm{N}(0,I)$ where I is identity matrix of size n?
If $x_i \sim \mathrm{N}(0,\sigma)$ for ...
31
votes
3answers
4k views
Brain-teaser: What is the expected length of an iid sequence that is monotonically increasing when drawn from a uniform [0,1] distribution?
This is an interview question for a quantitative analyst position, reported here. Suppose we are drawing from a uniform $[0,1]$ distribution and the draws are iid, what is the expected length of a ...
1
vote
1answer
98 views
How do I create an iid Rademacher sequence?
The lecture notes say:
Let $(\Omega,\mathcal{A},P) = ((0,1],\mathcal{B}((0,1]),\lambda)$ where $\lambda$ is the Lebesgue measure on the unit interval.
Define $X(\omega) = 1$ for $\omega > 1/2$ ...
