Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
156
questions
0
votes
1answer
27 views
When using a gaussian link in GLM, what are the assumptions?
In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
0answers
15 views
How to compute ESS (Effective Sample Size)?
I implemented the ESS calculation according to this manual like this:
...
1
vote
0answers
41 views
What if my samples are not IIDs? [closed]
I'm testing my system and getting some results over time (Queues size, wating time, ecc.). When I plot the correlograms they exhibit a high degree of autocorrelation, even for high lags. Now, what are ...
0
votes
0answers
28 views
What exactly is p(x,y) in the context of iid assumption in machine learning?
In machine learning iid assumption means that examples in the dataset are independent and drawn from the same probability distribution (i.e., identically distributed).
Here, the probability ...
10
votes
3answers
882 views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
1
vote
0answers
36 views
What problems happen when $L(\theta|x_1,x_2,…,x_n)$ be a function of differently distributed random variables?
Given $L(\theta|x_1,x_2,...,x_n)$, may $x_1,x_2,...,x_n$ not all be identically distributed? What problems happen in the not identically distributed case compared to the iid case?
1
vote
0answers
18 views
Shouldn’t we say independent given the distribution?
In statistics we often deal with iid random variables: independent identically distributed. But if we don’t know the distribution (say we still know the support is {0, 1}), and we get a sample x1, say ...
2
votes
1answer
69 views
Why is the MLE for variance in single linear regression biased? [duplicate]
I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):
...
1
vote
0answers
70 views
Probability that one positive random variable is greater than another independent and identically distributed positive random variable
Suppose I have the following random variables $X_{0} \sim f(x)$ and $X_{1} \sim f(x)$ are independent. I want to know the probability that $X_{0} > X_{1}$. So I think I want to find:
$$ P(X_{0}>...
2
votes
1answer
100 views
Probability of a machine failing when components fail independently
Our machine is made out of 3634 units of component A which has failure rate of 10%, 1656 units of component B with failure rate of 35% and 3368 units of component C with failure rate of 55%. Assuming ...
1
vote
1answer
207 views
UCB Exploration in Reinforcement Learning
I have two questions regarding the upper confidence bounds (UCB) exploration in reinforcement learning:
UCB exploration is derived from Hoeffding's inequality which assumes that the reward is bounded ...
0
votes
0answers
36 views
Bootstrapping regression coefficient for time series
One of the fundamental assumptions of bootstrap is that the samples are independent and identically distributed (i.i.d). This is the reason why it is difficult to bootstrap time-series because the ...
1
vote
2answers
122 views
Random Variable with IID always Gaussian?
Is there a case when we assume a random variable $\epsilon$ to be IID and assume its distribution is not gaussian?
0
votes
0answers
26 views
How to check Gauß-Markov theorem after OLS estimation?
I have estimate a simple bivariate regression with OLS and want to proof if the estimator is unbiased.
I found that the Gauß-Markov theorem consists of 4 Assumptions and the first 3 can be rewrite as ...
2
votes
1answer
76 views
Expectation Value of a Product of Many IID variables
First of all, I apologize for not being rigorous, but I am not a statistitian by background.
Imagine you have $N$ i.i.d. positive random variables $X_1...X_N$ and you are trying to compute a ...
0
votes
2answers
120 views
Check if residuals are IID (timeseries)
How can I check in R after decomposing a time series if my residual component is IID noise?
Would this be the best way (to use the autocorrelation function) and check for 0 correlation on all lags > ...
4
votes
0answers
193 views
What is the fundamental assumption for using resampling methods?
Suppose that I observe a set of non-i.i.d. data (time series) $\mathcal{L} = \left\lbrace (y_{t}, x_{t}) \right\rbrace_{t=1}^{T}$ with $x_{t} = (x_{t1},\ldots,x_{tP})$ a real valued vector of $P$ ...
1
vote
0answers
28 views
When does $E[f(X_i)]=E[f(X_j)], i\neq j$?
Suppose we have random variables $X_1, \dots, X_N$, with joint probability distribution $F_{X_1,\dots,X_N}$. Under what conditions does the following equality holds?
$$E[f(X_i)]=E[f(X_j)],\ \ i\neq ...
6
votes
2answers
297 views
Expectation of inverse of sum of positive iid variables
Let $(X_i)_{i}$ be a sequence of iid positive variables of mean 1 and variance $\sigma^2$. Let $\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$.
My question is: Can we can bound $\mathbb{E}(1/\bar{X}_n)$ as ...
0
votes
1answer
209 views
Are i.i.d. random variables always associated with a distribution function? [closed]
It is known that independent and identically distributed (i.i.d.) random variables are mutually independent and each random variable has the same probability distribution as the others.
However, is ...
1
vote
1answer
46 views
Implications of i.i.d. sample
I have the following question:
I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as
$\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
43 views
Are feature values random variable realizations, or random variables themselves?
I'm new to ML/stats so got confused with what I supposed was simple notation.
For simplicity, say I have a data set with just one column:
From probabilistic perspective, I had understood that X ...
6
votes
1answer
77 views
Is MLE intrinsically connected to logs?
My mathematical exploration led me the following claim:
Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
1
vote
0answers
35 views
Functional data analysis references
I am working on a inferential framework for functional data. I have categorical independent variables and a functional response variables. Different implementations of functional linear models are ...
3
votes
1answer
156 views
Does joint IID imply marginal IID?
Suppose you have a 2 dimension random vector denoted $(X,Y)$ that is Independent and Identically Distributed (IID) for a sequence of draws $((X_1,Y_1),(X_2,Y_2),...,(X_n,Y_n))$. Does this imply that ...
7
votes
3answers
152 views
Calculate the variance of $\sum\limits_{i=1}^{n-1} \sum\limits_{j=i+1}^n S(X_i - X_j)$ for $X_1,\ldots,X_n$ i.i.d. random variables
In p.88 of Wand & Jones (1995), they asked to show the following result.
Let $X_1,\ldots,X_n$ be a set of i.i.d. random variables and define $$U=2n^{-2}\sum_{i=1}^{n-1} \sum_{j=i+1}^n S(X_i - ...
5
votes
2answers
225 views
Identically distributed vs P(X > Y) = P(Y > X)
I've two related propositions which seem correct intuitively, but I struggle to prove them properly.
Question 1
Prove or disprove: If $X$ and $Y$ are independent and have identical marginal ...
1
vote
0answers
21 views
What final test to apply to a permutation test (by compression) algorithm?
I'm following in the steps of §5.1 Permutation Testing, NIST Special Publication 800-90B, Recommendation for the Entropy Sources Used for Random Bit Generation.
I'm developing my test for ...
0
votes
1answer
77 views
Likelihood when points aren't i.i.d
If we assume that we have a set of N data points given as $\textbf{X}$ and corresponding targets vectors $\textbf{T}$, where both represents matrices in this case. For an i.i.d we could write the ...
1
vote
0answers
82 views
Estimate unknown sum of iid random variables
Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known.
...
3
votes
1answer
135 views
Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID
From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
0answers
151 views
Expectation of product
Let $\{X_i\}_{i\in I}$ be a finite collection of i.i.d random variables. I have found that $$E[\prod_i X_i]=\prod_i E[X_i]$$ But I haven't found a proof of this fact. What is the proof?
1
vote
0answers
28 views
Minimum of i.i.d. Random Variables [duplicate]
What importance does the minimum of an identical and independently distributed random variables play in probability distribution?
1
vote
1answer
112 views
Role of random sample assumption in consistency of OLS estimator
I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it?
I am confused because for example I have ...
4
votes
1answer
53 views
A symmetric iid process
Let $X_1, X_2, \ldots$ be an iid process with $X_i$ having a symmetric distribution around $0$. Then can I always write
$$X_1 - \alpha X_{t-1}-\alpha^2 X_{t-2}-\cdots \stackrel{iid}{=} X_1 + |\alpha| ...
1
vote
1answer
134 views
Covariance between a variable and a non-linear transformation of it
Suppose $\epsilon \overset{\text{iid}}{\sim} N(0, \sigma^2)$
Can we make any assumptions about Cov$(\epsilon, \frac{\epsilon^2}{1 + \epsilon^2})$?
1
vote
0answers
65 views
Can i say an iid process with $0$ mean has homogeneous independent increment?
I think the title is itself self-explanatory. My idea says that an IID process with $0$ mean has to have independent increment. But, I do not understand how to prove homogeneity?
4
votes
2answers
74 views
Distribution of EMA of iid $\chi_1^2$
I am wondering if it is possible to get an analytical expression for the distribution of a random variable defined as
$$
r = \sum_{k=0}^\infty a_i b^i
$$
where $a_i$ are iid chi-square variable ...
0
votes
0answers
30 views
Wind (speed) - a non-iid random variable?
Over a time period of 1 year, half hourly wind speed data is collected. Is it false to state that the collected data is not-iid? I know that wind is seen as a non-stationary process, but does it also ...
1
vote
1answer
29 views
Samples drawn(iid) from univariate gaussian. Does their combination is drawn from multivariate gaussian
If $x_i \sim \mathrm{N}(0,1)$ for i = 1:n and $x_i$ are iid, is it true that $(x_1, x_2, ...,x_{n}) \sim \mathrm{N}(0,I)$ where I is identity matrix of size n?
If $x_i \sim \mathrm{N}(0,\sigma)$ for ...
30
votes
3answers
3k views
Brain-teaser: What is the expected length of an iid sequence that is monotonically increasing when drawn from a uniform [0,1] distribution?
This is an interview question for a quantitative analyst position, reported here. Suppose we are drawing from a uniform $[0,1]$ distribution and the draws are iid, what is the expected length of a ...
1
vote
1answer
91 views
How do I create an iid Rademacher sequence?
The lecture notes say:
Let $(\Omega,\mathcal{A},P) = ((0,1],\mathcal{B}((0,1]),\lambda)$ where $\lambda$ is the Lebesgue measure on the unit interval.
Define $X(\omega) = 1$ for $\omega > 1/2$ ...
5
votes
0answers
112 views
Finite sum of beta prime iid random variables
The beta prime distribution is infinitely divisible, as proved in Steutel and van Harn, 2003 (Appendix B). Sadly, in this book, there is no espression of the parameters of the distribution of n ...
1
vote
1answer
22 views
Estimating the mutual information in high dimension when all but one variable are iid
I have a function $f(x_{1},\dots,x_{n})$ where $n$ is large and I would like to estimate the mutual information between the random variable $f(X_{1},\dots,X_{n})$ and the independent and identically ...
9
votes
2answers
263 views
Uniform PDF of the difference of two r.v
Is it possible to have the PDF of the difference of two iid r.v.'s look like a rectangle (instead of, say, the triangle we get if the r.v.'s are taken from the uniform distribution).
i.e. is it ...
5
votes
3answers
199 views
Generating identically-distributed random variables with a constraint
Is there a way to generate identically-distributed random variables (eg $x_1,x_2,x_3,x_4$) with the following constraint:
$\frac{x_1*x_2}{x_3*x_4} ≡ 1$
$x \in (0,1)$
Please note that simply ...
4
votes
2answers
877 views
Exchangeability and IID random variables
Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the ...
3
votes
1answer
584 views
IID in real life /Machine Learning - When is data truly IID?
In a course I am studying at Berkeley,some student said about a particular Dataset "Data is not iid" and the lecturer agreed with him.
https://youtu.be/kl_G95uKTHw?list=...
2
votes
1answer
101 views
Finding $\mathrm{Var}(N)$ if $N=\inf\{n\ge1:\sum_{i=1}^nX_i>1\}$ where $X_i$'s are i.i.d Exponential variables
Suppose $X_1, X_2, X_3, \ldots, X_n$ be independent and identically distributed random variables having an exponential distribution with mean $\frac{1}{\lambda}$.
If $S_n = X_1 + X_2 + \ldots + X_n$ ...
1
vote
2answers
2k views
Simulating data for linear regression
I am trying to simulate two data set for multiple linear regression. I want one data which is independent and identically distributed and the other is not. So far, I have done the following:
...
