Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
5
votes
3answers
99 views
Calculate the variance of $\sum\limits_{i=1}^{n-1} \sum\limits_{j=i+1}^n S(X_i - X_j)$ for $X_1,\ldots,X_n$ i.i.d. random variables
In p.88 of Wand & Jones (1995), they asked to show the following result.
Let $X_1,\ldots,X_n$ be a set of i.i.d. random variables and define $$U=2n^{-2}\sum_{i=1}^{n-1} \sum_{j=i+1}^n S(X_i - ...
5
votes
2answers
123 views
Identically distributed vs P(X > Y) = P(Y > X)
I've two related propositions which seem correct intuitively, but I struggle to prove them properly.
Question 1
Prove or disprove: If $X$ and $Y$ are independent and have identical marginal ...
0
votes
0answers
7 views
What final test to apply to a permutation test (by compression) algorithm?
I'm following in the steps of §5.1 Permutation Testing, NIST Special Publication 800-90B, Recommendation for the Entropy Sources Used for Random Bit Generation. I can't link to this due to the ...
0
votes
1answer
38 views
Likelihood when points aren't i.i.d
If we assume that we have a set of N data points given as $\textbf{X}$ and corresponding targets vectors $\textbf{T}$, where both represents matrices in this case. For an i.i.d we could write the ...
1
vote
0answers
30 views
Estimate unknown sum of iid random variables
Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known.
...
2
votes
1answer
35 views
Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID
From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
0answers
44 views
Expectation of product
Let $\{X_i\}_{i\in I}$ be a finite collection of i.i.d random variables. I have found that $$E[\prod_i X_i]=\prod_i E[X_i]$$ But I haven't found a proof of this fact. What is the proof?
1
vote
0answers
22 views
Minimum of i.i.d. Random Variables [duplicate]
What importance does the minimum of an identical and independently distributed random variables play in probability distribution?
1
vote
1answer
46 views
Role of random sample assumption in consistency of OLS estimator
I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it?
I am confused because for example I have ...
2
votes
0answers
32 views
A symmetric iid process
Let $X_1, X_2, \ldots$ be an iid process with $X_i$ having a symmetric distribution around $0$. Then can I always write
$$X_1 - \alpha X_{t-1}-\alpha^2 X_{t-2}-\cdots \stackrel{iid}{=} X_1 + |\alpha| ...
0
votes
1answer
32 views
Covariance between a variable and a non-linear transformation of it
Suppose $\epsilon \overset{\text{iid}}{\sim} N(0, \sigma^2)$
Can we make any assumptions about Cov$(\epsilon, \frac{\epsilon^2}{1 + \epsilon^2})$?
1
vote
0answers
47 views
Can i say an iid process with $0$ mean has homogeneous independent increment?
I think the title is itself self-explanatory. My idea says that an IID process with $0$ mean has to have independent increment. But, I do not understand how to prove homogeneity?
4
votes
2answers
69 views
Distribution of EMA of iid $\chi_1^2$
I am wondering if it is possible to get an analytical expression for the distribution of a random variable defined as
$$
r = \sum_{k=0}^\infty a_i b^i
$$
where $a_i$ are iid chi-square variable ...
0
votes
0answers
21 views
Wind (speed) - a non-iid random variable?
Over a time period of 1 year, half hourly wind speed data is collected. Is it false to state that the collected data is not-iid? I know that wind is seen as a non-stationary process, but does it also ...
1
vote
1answer
19 views
Samples drawn(iid) from univariate gaussian. Does their combination is drawn from multivariate gaussian
If $x_i \sim \mathrm{N}(0,1)$ for i = 1:n and $x_i$ are iid, is it true that $(x_1, x_2, ...,x_{n}) \sim \mathrm{N}(0,I)$ where I is identity matrix of size n?
If $x_i \sim \mathrm{N}(0,\sigma)$ for ...
0
votes
0answers
23 views
Estimating EVT for non-i.i.d. data
I have a pnl time series (length more than 10 years) of a large diversified financial portfolio. on which i am trying to estimate VaR based on the method described in the paper : "Estimation of Tail-...
25
votes
2answers
2k views
Brain-teaser: What is the expected length of an iid sequence that is monotonically increasing when drawn from a uniform [0,1] distribution?
This is an interview question for a quantitative analyst position, reported here. Suppose we are drawing from a uniform $[0,1]$ distribution and the draws are iid, what is the expected length of a ...
1
vote
1answer
36 views
How do I create an iid Rademacher sequence?
The lecture notes say:
Let $(\Omega,\mathcal{A},P) = ((0,1],\mathcal{B}((0,1]),\lambda)$ where $\lambda$ is the Lebesgue measure on the unit interval.
Define $X(\omega) = 1$ for $\omega > 1/2$ ...
4
votes
0answers
48 views
Finite sum of beta prime iid random variables
The beta prime distribution is infinitely divisible, as proved in Steutel and van Harn, 2003 (Appendix B). Sadly, in this book, there is no espression of the parameters of the distribution of n ...
1
vote
1answer
18 views
Estimating the mutual information in high dimension when all but one variable are iid
I have a function $f(x_{1},\dots,x_{n})$ where $n$ is large and I would like to estimate the mutual information between the random variable $f(X_{1},\dots,X_{n})$ and the independent and identically ...
7
votes
2answers
141 views
Uniform PDF of the difference of two r.v
Is it possible to have the PDF of the difference of two iid r.v.'s look like a rectangle (instead of, say, the triangle we get if the r.v.'s are taken from the uniform distribution).
i.e. is it ...
5
votes
3answers
153 views
Generating identically-distributed random variables with a constraint
Is there a way to generate identically-distributed random variables (eg $x_1,x_2,x_3,x_4$) with the following constraint:
$\frac{x_1*x_2}{x_3*x_4} ≡ 1$
$x \in (0,1)$
Please note that simply ...
3
votes
2answers
280 views
Exchangeability and IID random variables
Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the ...
3
votes
1answer
263 views
IID in real life /Machine Learning - When is data truly IID?
In a course I am studying at Berkeley,some student said about a particular Dataset "Data is not iid" and the lecturer agreed with him.
https://youtu.be/kl_G95uKTHw?list=...
0
votes
0answers
15 views
Binomial regression on grouped data where the number of groups approaches the number of observations
Context: I am having trouble getting mixed or random effect models to converge, I think, due to the number groups compared with the number of observations.
Without being able to run proper mixed and ...
2
votes
0answers
53 views
Finding $\mathrm{Var}(N)$ if $N=\inf\{n\ge1:\sum_{i=1}^nX_i>1\}$ where $X_i$'s are i.i.d Exponential variables
Suppose $X_1, X_2, X_3, \ldots, X_n$ be independent and identically distributed random variables having an exponential distribution with mean $\frac{1}{\lambda}$.
If $S_n = X_1 + X_2 + \ldots + X_n$ ...
1
vote
2answers
544 views
Simulating data for linear regression
I am trying to simulate two data set for multiple linear regression. I want one data which is independent and identically distributed and the other is not. So far, I have done the following:
...
7
votes
1answer
90 views
Theory behind testing whether $\mu \in \mathbb{Q}$ for $X \sim \mathcal{N}(\mu, \sigma^2)$
Suppose that $X_i \stackrel{\mbox{i.i.d.}}{\sim} \mathcal{N} (\mu, \sigma^2)$, where $\sigma^2$ is known. Using this data, we wish to test whether $\mu \in \mathbb{Q}$, that is, whether the mean $\mu$ ...
4
votes
3answers
247 views
Why i.i.d. is the most conservative distribution assumption
I am reading Statistical Rethinking (Section 4.3).
When talking about the i.i.d. assumption used to build a linear regression model - without knowing if distribution values are correlated, the author ...
0
votes
0answers
24 views
iid time series through different time scales
How can I show that if a time serie is iid at the scale 1 day, it is also the case at the scale 2 days?
More precisely, I am considering financial returns.
I have iid daily returns $$r_{t,1d}=\frac{...
0
votes
1answer
380 views
Autocorrelation for iid noise
I am reading through the Introduction to Time Series and Forecasting Springer series textbook by Brockwell and Davis. On page 16 they discuss a simulated sequence of 200 iid normal random variables ...
1
vote
1answer
82 views
Convergence of gamma distribution
In this problem, X follows a gamma distribution with shape parameter 2 and scale parameter 1, the mean of such n independent and identical random variables should converge to 2. In my opinion, the ...
2
votes
2answers
140 views
Is independence assumption needed for Method of Moments estimator
I read about Method of Moments estimator (MOM) in Statistical Inference by Casella and Berger. In MOM description, I do not see the requirements that the sample should be iid. However, in the examples,...
0
votes
1answer
340 views
How to minimize $Variance(S)$?
Suppose $X_1,....,X_n$ are $iid$ random variables and for each of them
$Variance(X_i)= \sigma^2$. $a_1...a_n$ are also real numbers and
$\sum_{i=1}^n a_i = 1$ If $S = \sum_{i=1}^na_iX_i$, prove $...
2
votes
0answers
36 views
i.i.d. assumption for pairwise data generated from clusters
In the context of record linkage, data de-duplication, or entity resolution, we attempt to merge entities that refer to the same thing into a single object. The obvious example is an address database ...
1
vote
1answer
39 views
SEM-LPA model fit using BIC
I am studying a paper and while I have done a lot of additional reading, I would like some hints on on this issue as I am somewhat stuck.
My question is whether a Structural Equation Model of the ...
0
votes
1answer
65 views
Non-random sampling iid
Suppose I know that 40% of the population are single but my sampling procedure is non-random and skewed towards singles in a way such that in my sample 60% are single. So my data would not be i.i.d. ...
6
votes
2answers
205 views
$X$, $Y$ independent identically distributed. Are there counterexamples to symmetry of $X-Y$?
That $X-Y$ should be symmetrically distributed for iid $X,Y$ is obvious simply by interchanging the roles of $X$ and $Y$ -- informally we might argue
Let $Z=X-Y$ have distribution $F$. The roles of ...
0
votes
0answers
103 views
$P(X+Y>Z)$ given $X,Y,Z$ are i.i.d random variables
Given $X, Y, Z$ i.i.d random variables the probability $P(X+Y>Z)$ can be found by the following three approaches:
$X+Y-Z > 0$ region cuts the $X Y Z$ volume into two equal volumes as $X+Y-Z=0$ ...
4
votes
1answer
51 views
Distribution notation
beginner here.
I've seen in a paper $x\sim NIID(0,1)$ and $y\sim N(0,1)$.
This confuses me: to me these both look like standard Normal variables. Can someone explain the difference?
0
votes
1answer
78 views
When you have two of the same distribution, are they necessarily i.i.d.?
When you have two of the same distribution, are they necessarily independent and identically distributed (i.i.d.)?
For example, when you have two from the normal distribution with the same ...
1
vote
1answer
365 views
What are possible effects of adding a variable to a multiple-regression model?
I read that when adding a irrelevant variable to a multiple regression, the variance of the other regression coefficients will increase.
However, does adding a relevant variable (to combat omitted ...
1
vote
0answers
178 views
Which transformations can be performed on data that maintain the i.i.d. property of a sample?
I have problems to understand how data transformations interact with the property of samples being i.i.d. Especially when nontrivial operations are performed on each element of the sample, I am not ...
6
votes
3answers
474 views
Variance of Minimum and Maximum of 2 iid Normal
Let $X$ and $Y$ be iid $\sim Normal(0,1)$
Let $A=max(X,Y)$ and $B=min(X,Y)$
What are $Var(A)$ and $Var(B)$?
From simulation, I get $Var(A)=Var(B)$ approximately 0.70.
How do I get this ...
0
votes
1answer
291 views
Independent and identically distributed samples
Say i am training a neural network and have 10 samples with 4 variables each and 1 label assigned to each observation. What does it mean to say that the samples are independent and identically ...
4
votes
1answer
142 views
If X and Y both have the same pdf, they are identically distributed, but can I say that they are independent? [duplicate]
More specific: I have two variables, X and Y, they are exponentially distributed with parameter a. I know that the distribution of X + Y follows a Gamma(2,a) distribution IF they are independent (I ...
2
votes
0answers
115 views
“Principal component analysis has its nice properties under the assumption of i.i.d. observations. ”
I recently discovered this post "Can Principal Component Analysis be used on stock prices / non-stationary data?" which said "Principal component analysis has its nice properties under the assumption ...
0
votes
2answers
54 views
Conditionally identical distributions
This question arose from other (now deleted) question that quoted paper saying something similar to what is described.
Given
$$
\mu \sim \mathcal{N}(\mu_0, \sigma_0^2)\\
X \sim \mathcal{N}(\mu, \...
-1
votes
2answers
1k views
Why white noise process and IID process are considered martingale [closed]
Can anyone explain to me why A white noise process (εt) and An IID process (εt) are martingales?
1
vote
0answers
44 views
If some residuals in my regression look serially correlated, do I need to discard all the corresponding data points or not?
I'm trying to build a regression from the results of a campaign of physical experiments, consisting of 7 different test campaigns. In each test campaign, all test points are repeated two times, once ...
