Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
0
votes
0answers
27 views
When does $E[f(X_i)]=E[f(X_j)], i\neq j$?
Suppose we have random variables $X_1, \dots, X_N$, with joint probability distribution $F_{X_1,\dots,X_N}$. Under what conditions does the following equality holds?
$$E[f(X_i)]=E[f(X_j)],\ \ i\neq ...
6
votes
2answers
167 views
Expectation of inverse of sum of positive iid variables
Let $(X_i)_{i}$ be a sequence of iid positive variables of mean 1 and variance $\sigma^2$. Let $\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$.
My question is: Can we can bound $\mathbb{E}(1/\bar{X}_n)$ as ...
0
votes
1answer
126 views
Are i.i.d. random variables always associated with a distribution function? [closed]
It is known that independent and identically distributed (i.i.d.) random variables are mutually independent and each random variable has the same probability distribution as the others.
However, is ...
1
vote
1answer
33 views
Implications of i.i.d. sample
I have the following question:
I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as
$\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
34 views
Are feature values random variable realizations, or random variables themselves?
I'm new to ML/stats so got confused with what I supposed was simple notation.
For simplicity, say I have a data set with just one column:
From probabilistic perspective, I had understood that X ...
6
votes
1answer
66 views
Is MLE intrinsically connected to logs?
My mathematical exploration led me the following claim:
Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
1
vote
0answers
24 views
Functional data analysis references
I am working on a inferential framework for functional data. I have categorical independent variables and a functional response variables. Different implementations of functional linear models are ...
3
votes
1answer
65 views
Does joint IID imply marginal IID?
Suppose you have a 2 dimension random vector denoted $(X,Y)$ that is Independent and Identically Distributed (IID) for a sequence of draws $((X_1,Y_1),(X_2,Y_2),...,(X_n,Y_n))$. Does this imply that ...
6
votes
3answers
120 views
Calculate the variance of $\sum\limits_{i=1}^{n-1} \sum\limits_{j=i+1}^n S(X_i - X_j)$ for $X_1,\ldots,X_n$ i.i.d. random variables
In p.88 of Wand & Jones (1995), they asked to show the following result.
Let $X_1,\ldots,X_n$ be a set of i.i.d. random variables and define $$U=2n^{-2}\sum_{i=1}^{n-1} \sum_{j=i+1}^n S(X_i - ...
5
votes
2answers
170 views
Identically distributed vs P(X > Y) = P(Y > X)
I've two related propositions which seem correct intuitively, but I struggle to prove them properly.
Question 1
Prove or disprove: If $X$ and $Y$ are independent and have identical marginal ...
0
votes
0answers
10 views
What final test to apply to a permutation test (by compression) algorithm?
I'm following in the steps of §5.1 Permutation Testing, NIST Special Publication 800-90B, Recommendation for the Entropy Sources Used for Random Bit Generation. I can't link to this due to the ...
0
votes
1answer
50 views
Likelihood when points aren't i.i.d
If we assume that we have a set of N data points given as $\textbf{X}$ and corresponding targets vectors $\textbf{T}$, where both represents matrices in this case. For an i.i.d we could write the ...
1
vote
0answers
41 views
Estimate unknown sum of iid random variables
Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known.
...
3
votes
1answer
55 views
Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID
From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
1
vote
0answers
108 views
Expectation of product
Let $\{X_i\}_{i\in I}$ be a finite collection of i.i.d random variables. I have found that $$E[\prod_i X_i]=\prod_i E[X_i]$$ But I haven't found a proof of this fact. What is the proof?
1
vote
0answers
24 views
Minimum of i.i.d. Random Variables [duplicate]
What importance does the minimum of an identical and independently distributed random variables play in probability distribution?
1
vote
1answer
68 views
Role of random sample assumption in consistency of OLS estimator
I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it?
I am confused because for example I have ...
4
votes
1answer
44 views
A symmetric iid process
Let $X_1, X_2, \ldots$ be an iid process with $X_i$ having a symmetric distribution around $0$. Then can I always write
$$X_1 - \alpha X_{t-1}-\alpha^2 X_{t-2}-\cdots \stackrel{iid}{=} X_1 + |\alpha| ...
0
votes
1answer
66 views
Covariance between a variable and a non-linear transformation of it
Suppose $\epsilon \overset{\text{iid}}{\sim} N(0, \sigma^2)$
Can we make any assumptions about Cov$(\epsilon, \frac{\epsilon^2}{1 + \epsilon^2})$?
1
vote
0answers
55 views
Can i say an iid process with $0$ mean has homogeneous independent increment?
I think the title is itself self-explanatory. My idea says that an IID process with $0$ mean has to have independent increment. But, I do not understand how to prove homogeneity?
4
votes
2answers
72 views
Distribution of EMA of iid $\chi_1^2$
I am wondering if it is possible to get an analytical expression for the distribution of a random variable defined as
$$
r = \sum_{k=0}^\infty a_i b^i
$$
where $a_i$ are iid chi-square variable ...
0
votes
0answers
25 views
Wind (speed) - a non-iid random variable?
Over a time period of 1 year, half hourly wind speed data is collected. Is it false to state that the collected data is not-iid? I know that wind is seen as a non-stationary process, but does it also ...
1
vote
1answer
26 views
Samples drawn(iid) from univariate gaussian. Does their combination is drawn from multivariate gaussian
If $x_i \sim \mathrm{N}(0,1)$ for i = 1:n and $x_i$ are iid, is it true that $(x_1, x_2, ...,x_{n}) \sim \mathrm{N}(0,I)$ where I is identity matrix of size n?
If $x_i \sim \mathrm{N}(0,\sigma)$ for ...
0
votes
0answers
32 views
Estimating EVT for non-i.i.d. data
I have a pnl time series (length more than 10 years) of a large diversified financial portfolio. on which i am trying to estimate VaR based on the method described in the paper : "Estimation of Tail-...
27
votes
3answers
3k views
Brain-teaser: What is the expected length of an iid sequence that is monotonically increasing when drawn from a uniform [0,1] distribution?
This is an interview question for a quantitative analyst position, reported here. Suppose we are drawing from a uniform $[0,1]$ distribution and the draws are iid, what is the expected length of a ...
1
vote
1answer
48 views
How do I create an iid Rademacher sequence?
The lecture notes say:
Let $(\Omega,\mathcal{A},P) = ((0,1],\mathcal{B}((0,1]),\lambda)$ where $\lambda$ is the Lebesgue measure on the unit interval.
Define $X(\omega) = 1$ for $\omega > 1/2$ ...
4
votes
0answers
62 views
Finite sum of beta prime iid random variables
The beta prime distribution is infinitely divisible, as proved in Steutel and van Harn, 2003 (Appendix B). Sadly, in this book, there is no espression of the parameters of the distribution of n ...
1
vote
1answer
20 views
Estimating the mutual information in high dimension when all but one variable are iid
I have a function $f(x_{1},\dots,x_{n})$ where $n$ is large and I would like to estimate the mutual information between the random variable $f(X_{1},\dots,X_{n})$ and the independent and identically ...
7
votes
2answers
156 views
Uniform PDF of the difference of two r.v
Is it possible to have the PDF of the difference of two iid r.v.'s look like a rectangle (instead of, say, the triangle we get if the r.v.'s are taken from the uniform distribution).
i.e. is it ...
5
votes
3answers
165 views
Generating identically-distributed random variables with a constraint
Is there a way to generate identically-distributed random variables (eg $x_1,x_2,x_3,x_4$) with the following constraint:
$\frac{x_1*x_2}{x_3*x_4} ≡ 1$
$x \in (0,1)$
Please note that simply ...
3
votes
2answers
457 views
Exchangeability and IID random variables
Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the ...
3
votes
1answer
333 views
IID in real life /Machine Learning - When is data truly IID?
In a course I am studying at Berkeley,some student said about a particular Dataset "Data is not iid" and the lecturer agreed with him.
https://youtu.be/kl_G95uKTHw?list=...
2
votes
1answer
90 views
Finding $\mathrm{Var}(N)$ if $N=\inf\{n\ge1:\sum_{i=1}^nX_i>1\}$ where $X_i$'s are i.i.d Exponential variables
Suppose $X_1, X_2, X_3, \ldots, X_n$ be independent and identically distributed random variables having an exponential distribution with mean $\frac{1}{\lambda}$.
If $S_n = X_1 + X_2 + \ldots + X_n$ ...
1
vote
2answers
997 views
Simulating data for linear regression
I am trying to simulate two data set for multiple linear regression. I want one data which is independent and identically distributed and the other is not. So far, I have done the following:
...
7
votes
1answer
92 views
Theory behind testing whether $\mu \in \mathbb{Q}$ for $X \sim \mathcal{N}(\mu, \sigma^2)$
Suppose that $X_i \stackrel{\mbox{i.i.d.}}{\sim} \mathcal{N} (\mu, \sigma^2)$, where $\sigma^2$ is known. Using this data, we wish to test whether $\mu \in \mathbb{Q}$, that is, whether the mean $\mu$ ...
4
votes
3answers
304 views
Why i.i.d. is the most conservative distribution assumption
I am reading Statistical Rethinking (Section 4.3).
When talking about the i.i.d. assumption used to build a linear regression model - without knowing if distribution values are correlated, the author ...
0
votes
0answers
26 views
iid time series through different time scales
How can I show that if a time serie is iid at the scale 1 day, it is also the case at the scale 2 days?
More precisely, I am considering financial returns.
I have iid daily returns $$r_{t,1d}=\frac{...
0
votes
1answer
556 views
Autocorrelation for iid noise
I am reading through the Introduction to Time Series and Forecasting Springer series textbook by Brockwell and Davis. On page 16 they discuss a simulated sequence of 200 iid normal random variables ...
1
vote
1answer
168 views
Convergence of gamma distribution
In this problem, X follows a gamma distribution with shape parameter 2 and scale parameter 1, the mean of such n independent and identical random variables should converge to 2. In my opinion, the ...
2
votes
2answers
218 views
Is independence assumption needed for Method of Moments estimator
I read about Method of Moments estimator (MOM) in Statistical Inference by Casella and Berger. In MOM description, I do not see the requirements that the sample should be iid. However, in the examples,...
0
votes
1answer
537 views
How to minimize $Variance(S)$?
Suppose $X_1,....,X_n$ are $iid$ random variables and for each of them
$Variance(X_i)= \sigma^2$. $a_1...a_n$ are also real numbers and
$\sum_{i=1}^n a_i = 1$ If $S = \sum_{i=1}^na_iX_i$, prove $...
2
votes
0answers
40 views
i.i.d. assumption for pairwise data generated from clusters
In the context of record linkage, data de-duplication, or entity resolution, we attempt to merge entities that refer to the same thing into a single object. The obvious example is an address database ...
1
vote
1answer
42 views
SEM-LPA model fit using BIC
I am studying a paper and while I have done a lot of additional reading, I would like some hints on on this issue as I am somewhat stuck.
My question is whether a Structural Equation Model of the ...
0
votes
1answer
76 views
Non-random sampling iid
Suppose I know that 40% of the population are single but my sampling procedure is non-random and skewed towards singles in a way such that in my sample 60% are single. So my data would not be i.i.d. ...
7
votes
2answers
327 views
$X$, $Y$ independent identically distributed. Are there counterexamples to symmetry of $X-Y$?
That $X-Y$ should be symmetrically distributed for iid $X,Y$ is obvious simply by interchanging the roles of $X$ and $Y$ -- informally we might argue
Let $Z=X-Y$ have distribution $F$. The roles of ...
0
votes
0answers
110 views
$P(X+Y>Z)$ given $X,Y,Z$ are i.i.d random variables
Given $X, Y, Z$ i.i.d random variables the probability $P(X+Y>Z)$ can be found by the following three approaches:
$X+Y-Z > 0$ region cuts the $X Y Z$ volume into two equal volumes as $X+Y-Z=0$ ...
4
votes
1answer
51 views
Distribution notation
beginner here.
I've seen in a paper $x\sim NIID(0,1)$ and $y\sim N(0,1)$.
This confuses me: to me these both look like standard Normal variables. Can someone explain the difference?
0
votes
1answer
85 views
When you have two of the same distribution, are they necessarily i.i.d.?
When you have two of the same distribution, are they necessarily independent and identically distributed (i.i.d.)?
For example, when you have two from the normal distribution with the same ...
1
vote
1answer
442 views
What are possible effects of adding a variable to a multiple-regression model?
I read that when adding a irrelevant variable to a multiple regression, the variance of the other regression coefficients will increase.
However, does adding a relevant variable (to combat omitted ...
1
vote
0answers
189 views
Which transformations can be performed on data that maintain the i.i.d. property of a sample?
I have problems to understand how data transformations interact with the property of samples being i.i.d. Especially when nontrivial operations are performed on each element of the sample, I am not ...
