Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [iid]

iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.

5
votes
3answers
99 views

Calculate the variance of $\sum\limits_{i=1}^{n-1} \sum\limits_{j=i+1}^n S(X_i - X_j)$ for $X_1,\ldots,X_n$ i.i.d. random variables

In p.88 of Wand & Jones (1995), they asked to show the following result. Let $X_1,\ldots,X_n$ be a set of i.i.d. random variables and define $$U=2n^{-2}\sum_{i=1}^{n-1} \sum_{j=i+1}^n S(X_i - ...
5
votes
2answers
123 views

Identically distributed vs P(X > Y) = P(Y > X)

I've two related propositions which seem correct intuitively, but I struggle to prove them properly. Question 1 Prove or disprove: If $X$ and $Y$ are independent and have identical marginal ...
0
votes
0answers
7 views

What final test to apply to a permutation test (by compression) algorithm?

I'm following in the steps of §5.1 Permutation Testing, NIST Special Publication 800-90B, Recommendation for the Entropy Sources Used for Random Bit Generation. I can't link to this due to the ...
0
votes
1answer
38 views

Likelihood when points aren't i.i.d

If we assume that we have a set of N data points given as $\textbf{X}$ and corresponding targets vectors $\textbf{T}$, where both represents matrices in this case. For an i.i.d we could write the ...
1
vote
0answers
30 views

Estimate unknown sum of iid random variables

Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known. ...
2
votes
1answer
35 views

Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID

From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
0answers
44 views

Expectation of product

Let $\{X_i\}_{i\in I}$ be a finite collection of i.i.d random variables. I have found that $$E[\prod_i X_i]=\prod_i E[X_i]$$ But I haven't found a proof of this fact. What is the proof?
1
vote
0answers
22 views

Minimum of i.i.d. Random Variables [duplicate]

What importance does the minimum of an identical and independently distributed random variables play in probability distribution?
1
vote
1answer
46 views

Role of random sample assumption in consistency of OLS estimator

I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it? I am confused because for example I have ...
2
votes
0answers
32 views

A symmetric iid process

Let $X_1, X_2, \ldots$ be an iid process with $X_i$ having a symmetric distribution around $0$. Then can I always write $$X_1 - \alpha X_{t-1}-\alpha^2 X_{t-2}-\cdots \stackrel{iid}{=} X_1 + |\alpha| ...
0
votes
1answer
32 views

Covariance between a variable and a non-linear transformation of it

Suppose $\epsilon \overset{\text{iid}}{\sim} N(0, \sigma^2)$ Can we make any assumptions about Cov$(\epsilon, \frac{\epsilon^2}{1 + \epsilon^2})$?
1
vote
0answers
47 views

Can i say an iid process with $0$ mean has homogeneous independent increment?

I think the title is itself self-explanatory. My idea says that an IID process with $0$ mean has to have independent increment. But, I do not understand how to prove homogeneity?
4
votes
2answers
69 views

Distribution of EMA of iid $\chi_1^2$

I am wondering if it is possible to get an analytical expression for the distribution of a random variable defined as $$ r = \sum_{k=0}^\infty a_i b^i $$ where $a_i$ are iid chi-square variable ...
0
votes
0answers
21 views

Wind (speed) - a non-iid random variable?

Over a time period of 1 year, half hourly wind speed data is collected. Is it false to state that the collected data is not-iid? I know that wind is seen as a non-stationary process, but does it also ...
1
vote
1answer
19 views

Samples drawn(iid) from univariate gaussian. Does their combination is drawn from multivariate gaussian

If $x_i \sim \mathrm{N}(0,1)$ for i = 1:n and $x_i$ are iid, is it true that $(x_1, x_2, ...,x_{n}) \sim \mathrm{N}(0,I)$ where I is identity matrix of size n? If $x_i \sim \mathrm{N}(0,\sigma)$ for ...
0
votes
0answers
23 views

Estimating EVT for non-i.i.d. data

I have a pnl time series (length more than 10 years) of a large diversified financial portfolio. on which i am trying to estimate VaR based on the method described in the paper : "Estimation of Tail-...
25
votes
2answers
2k views

Brain-teaser: What is the expected length of an iid sequence that is monotonically increasing when drawn from a uniform [0,1] distribution?

This is an interview question for a quantitative analyst position, reported here. Suppose we are drawing from a uniform $[0,1]$ distribution and the draws are iid, what is the expected length of a ...
1
vote
1answer
36 views

How do I create an iid Rademacher sequence?

The lecture notes say: Let $(\Omega,\mathcal{A},P) = ((0,1],\mathcal{B}((0,1]),\lambda)$ where $\lambda$ is the Lebesgue measure on the unit interval. Define $X(\omega) = 1$ for $\omega > 1/2$ ...
4
votes
0answers
48 views

Finite sum of beta prime iid random variables

The beta prime distribution is infinitely divisible, as proved in Steutel and van Harn, 2003 (Appendix B). Sadly, in this book, there is no espression of the parameters of the distribution of n ...
1
vote
1answer
18 views

Estimating the mutual information in high dimension when all but one variable are iid

I have a function $f(x_{1},\dots,x_{n})$ where $n$ is large and I would like to estimate the mutual information between the random variable $f(X_{1},\dots,X_{n})$ and the independent and identically ...
7
votes
2answers
141 views

Uniform PDF of the difference of two r.v

Is it possible to have the PDF of the difference of two iid r.v.'s look like a rectangle (instead of, say, the triangle we get if the r.v.'s are taken from the uniform distribution). i.e. is it ...
5
votes
3answers
153 views

Generating identically-distributed random variables with a constraint

Is there a way to generate identically-distributed random variables (eg $x_1,x_2,x_3,x_4$) with the following constraint: $\frac{x_1*x_2}{x_3*x_4} ≡ 1$ $x \in (0,1)$ Please note that simply ...
3
votes
2answers
280 views

Exchangeability and IID random variables

Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the ...
3
votes
1answer
263 views

IID in real life /Machine Learning - When is data truly IID?

In a course I am studying at Berkeley,some student said about a particular Dataset "Data is not iid" and the lecturer agreed with him. https://youtu.be/kl_G95uKTHw?list=...
0
votes
0answers
15 views

Binomial regression on grouped data where the number of groups approaches the number of observations

Context: I am having trouble getting mixed or random effect models to converge, I think, due to the number groups compared with the number of observations. Without being able to run proper mixed and ...
2
votes
0answers
53 views

Finding $\mathrm{Var}(N)$ if $N=\inf\{n\ge1:\sum_{i=1}^nX_i>1\}$ where $X_i$'s are i.i.d Exponential variables

Suppose $X_1, X_2, X_3, \ldots, X_n$ be independent and identically distributed random variables having an exponential distribution with mean $\frac{1}{\lambda}$. If $S_n = X_1 + X_2 + \ldots + X_n$ ...
1
vote
2answers
544 views

Simulating data for linear regression

I am trying to simulate two data set for multiple linear regression. I want one data which is independent and identically distributed and the other is not. So far, I have done the following: ...
7
votes
1answer
90 views

Theory behind testing whether $\mu \in \mathbb{Q}$ for $X \sim \mathcal{N}(\mu, \sigma^2)$

Suppose that $X_i \stackrel{\mbox{i.i.d.}}{\sim} \mathcal{N} (\mu, \sigma^2)$, where $\sigma^2$ is known. Using this data, we wish to test whether $\mu \in \mathbb{Q}$, that is, whether the mean $\mu$ ...
4
votes
3answers
247 views

Why i.i.d. is the most conservative distribution assumption

I am reading Statistical Rethinking (Section 4.3). When talking about the i.i.d. assumption used to build a linear regression model - without knowing if distribution values are correlated, the author ...
0
votes
0answers
24 views

iid time series through different time scales

How can I show that if a time serie is iid at the scale 1 day, it is also the case at the scale 2 days? More precisely, I am considering financial returns. I have iid daily returns $$r_{t,1d}=\frac{...
0
votes
1answer
380 views

Autocorrelation for iid noise

I am reading through the Introduction to Time Series and Forecasting Springer series textbook by Brockwell and Davis. On page 16 they discuss a simulated sequence of 200 iid normal random variables ...
1
vote
1answer
82 views

Convergence of gamma distribution

In this problem, X follows a gamma distribution with shape parameter 2 and scale parameter 1, the mean of such n independent and identical random variables should converge to 2. In my opinion, the ...
2
votes
2answers
140 views

Is independence assumption needed for Method of Moments estimator

I read about Method of Moments estimator (MOM) in Statistical Inference by Casella and Berger. In MOM description, I do not see the requirements that the sample should be iid. However, in the examples,...
0
votes
1answer
340 views

How to minimize $Variance(S)$?

Suppose $X_1,....,X_n$ are $iid$ random variables and for each of them $Variance(X_i)= \sigma^2$. $a_1...a_n$ are also real numbers and $\sum_{i=1}^n a_i = 1$ If $S = \sum_{i=1}^na_iX_i$, prove $...
2
votes
0answers
36 views

i.i.d. assumption for pairwise data generated from clusters

In the context of record linkage, data de-duplication, or entity resolution, we attempt to merge entities that refer to the same thing into a single object. The obvious example is an address database ...
1
vote
1answer
39 views

SEM-LPA model fit using BIC

I am studying a paper and while I have done a lot of additional reading, I would like some hints on on this issue as I am somewhat stuck. My question is whether a Structural Equation Model of the ...
0
votes
1answer
65 views

Non-random sampling iid

Suppose I know that 40% of the population are single but my sampling procedure is non-random and skewed towards singles in a way such that in my sample 60% are single. So my data would not be i.i.d. ...
6
votes
2answers
205 views

$X$, $Y$ independent identically distributed. Are there counterexamples to symmetry of $X-Y$?

That $X-Y$ should be symmetrically distributed for iid $X,Y$ is obvious simply by interchanging the roles of $X$ and $Y$ -- informally we might argue Let $Z=X-Y$ have distribution $F$. The roles of ...
0
votes
0answers
103 views

$P(X+Y>Z)$ given $X,Y,Z$ are i.i.d random variables

Given $X, Y, Z$ i.i.d random variables the probability $P(X+Y>Z)$ can be found by the following three approaches: $X+Y-Z > 0$ region cuts the $X Y Z$ volume into two equal volumes as $X+Y-Z=0$ ...
4
votes
1answer
51 views

Distribution notation

beginner here. I've seen in a paper $x\sim NIID(0,1)$ and $y\sim N(0,1)$. This confuses me: to me these both look like standard Normal variables. Can someone explain the difference?
0
votes
1answer
78 views

When you have two of the same distribution, are they necessarily i.i.d.?

When you have two of the same distribution, are they necessarily independent and identically distributed (i.i.d.)? For example, when you have two from the normal distribution with the same ...
1
vote
1answer
365 views

What are possible effects of adding a variable to a multiple-regression model?

I read that when adding a irrelevant variable to a multiple regression, the variance of the other regression coefficients will increase. However, does adding a relevant variable (to combat omitted ...
1
vote
0answers
178 views

Which transformations can be performed on data that maintain the i.i.d. property of a sample?

I have problems to understand how data transformations interact with the property of samples being i.i.d. Especially when nontrivial operations are performed on each element of the sample, I am not ...
6
votes
3answers
474 views

Variance of Minimum and Maximum of 2 iid Normal

Let $X$ and $Y$ be iid $\sim Normal(0,1)$ Let $A=max(X,Y)$ and $B=min(X,Y)$ What are $Var(A)$ and $Var(B)$? From simulation, I get $Var(A)=Var(B)$ approximately 0.70. How do I get this ...
0
votes
1answer
291 views

Independent and identically distributed samples

Say i am training a neural network and have 10 samples with 4 variables each and 1 label assigned to each observation. What does it mean to say that the samples are independent and identically ...
4
votes
1answer
142 views

If X and Y both have the same pdf, they are identically distributed, but can I say that they are independent? [duplicate]

More specific: I have two variables, X and Y, they are exponentially distributed with parameter a. I know that the distribution of X + Y follows a Gamma(2,a) distribution IF they are independent (I ...
2
votes
0answers
115 views

“Principal component analysis has its nice properties under the assumption of i.i.d. observations. ”

I recently discovered this post "Can Principal Component Analysis be used on stock prices / non-stationary data?" which said "Principal component analysis has its nice properties under the assumption ...
0
votes
2answers
54 views

Conditionally identical distributions

This question arose from other (now deleted) question that quoted paper saying something similar to what is described. Given $$ \mu \sim \mathcal{N}(\mu_0, \sigma_0^2)\\ X \sim \mathcal{N}(\mu, \...
-1
votes
2answers
1k views

Why white noise process and IID process are considered martingale [closed]

Can anyone explain to me why A white noise process (εt) and An IID process (εt) are martingales?
1
vote
0answers
44 views

If some residuals in my regression look serially correlated, do I need to discard all the corresponding data points or not?

I'm trying to build a regression from the results of a campaign of physical experiments, consisting of 7 different test campaigns. In each test campaign, all test points are repeated two times, once ...