Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
250
questions
0
votes
1
answer
123
views
Expected value of iid squared conditioned on sum
I would be interested in finding the value of the following expression:
$$\mathbb{E}[X_k^2\mid S_N]$$
where $X_k$ are iid random variables with $\mathbb{E}[X_k]=\mu$ and $\operatorname{Var}[X_k]=\...
- 91
0
votes
0
answers
47
views
Are $X_i \varepsilon_i$ iid?
Take:
$X_i , \ i = 1, ... , n $ iid.
$\varepsilon_i , \ i = 1, ... ,n$ also iid.
$X_i \not \perp \varepsilon_j$ (they are not necessarily independent)
Are $X_i \varepsilon_i$ iid ?
- 1
4
votes
1
answer
304
views
Link between Cross-entropy and MLE
There are numerous material that show the relationship between MLE and cross-entropy.
Typically, these are the steps taken to show the relationship for a I.I.D data generating process $D = (X,Y)$:
$$
...
- 143
10
votes
5
answers
1k
views
Why GLM don't have an error term and why shouldn't residuals be i.i.d?
I've read dozens on post on the subject but I cannot figure this out. From what I've gathered, GLMS don't include an error term in their formulation unlike linear models (LM). I was wondering why (or ...
0
votes
0
answers
51
views
How to understand intuitively the CDF formula for the maximum statistic of three iid rv’s? [duplicate]
Given that all three iid rv’s ($X_1, X_2, X_3$) have CDF $F(x)$, the formula for the CDF $G(y)$ of the largest rv ($Y=X_i$) among the three is:
$G(y)=P(X_1 \leq y) \cdot P(X_2 \leq y) \cdot P(X_3 \leq ...
0
votes
0
answers
23
views
Method-of-moment of n IID random variables
The method-of-moment of $\sigma$ for the following pdf is
$$
\text{pdf}(x,\sigma) = \frac{x}{\sigma^2}\exp(-\frac{1}{2}\frac{x^2}{\sigma^2})
$$
$$
E[x] = \int_{0}^{\infty}\frac{x^2}{\sigma^2}\exp(-\...
1
vote
0
answers
29
views
Is it possible to describe repeating data patterns as a stochastic process?
Generally, can repetitive patterns in sensor readings (e.g. temperature measurements at different locations over time) be seen as some kind of stochastic process?
That is, if similar patterns repeat ...
- 161
1
vote
1
answer
75
views
Is the Distribution of Survival Times always IID?
I was reading about an approach to Survival Analysis called "First Hitting Time Models" (threshold regression): https://www.jstatsoft.org/article/view/v066i08 , Can Survival Models model the ...
0
votes
0
answers
63
views
Misunderstanding on the use of Popoviciu and von Szokefalvi Nagy's inequalities on the variance of a unbiased estimator
Let $X_1,\cdots,X_n$ be (discrete in my case) i.i.d. and bounded between $m$ and $M$. I'm interested in bounding the variance of an unbiased estimator:
$$\mathbb{V}\left[\frac1n\sum_{i=1}^nX_i\right]$$...
- 103
1
vote
0
answers
93
views
If $X_1, \dots, X_n$ iid, are $f(X_1), \dots, f(X_n)$, also iid? [duplicate]
If I have independent and identically distributed random variables $X_1, \dots, X_n$, then are $f(X_1), \dots, f(X_n)$ themselves independent and identically distributed?
I think the answer is yes, ...
- 31
1
vote
1
answer
178
views
Is a random walk i.i.d.? [closed]
Assuming that we have an AR(1) process:
$$X_t=\rho X_{t-1}+\varepsilon_t,\quad\varepsilon\sim IIDN(0,1)$$
And further assuming that $\rho=1$ and $X_0=0$ we have a random walk process:
$$X_t=\sum_{i=1}^...
- 55
0
votes
0
answers
129
views
How to find the MGF of the max of a set of i.i.d. exponential random variables
As the title suggests, I would like to find the MGF of the max of iid exponential random variables. Assume $Z=\max(x_{1},...,x_{n})$, where $x_{i}$ is distributed as exponential($\beta$) and has pdf $\...
0
votes
1
answer
53
views
About the requirement of iids to form a martingale
Most problems about martingales start with an assertion similar to the following one:
Suppose that $X_n$ are iid such that $\mathbb{E}[|X_n|]<\infty$ and $\mathbb{E}[X_n]=0$. Then $S_n = \sum_i^n ...
- 91
0
votes
1
answer
33
views
Are these trials identically distributed? These trials from a panel data are all Bernoulli experiments; do they have same probability of success p?
Scenario: I have an ordered/indexed sample that supposedly comes from a binomial distribution with fixed probability of success p. (By ordered/indexed I mean that not only I care about the percentage ...
- 9
1
vote
0
answers
28
views
iid condition of SPC
The control chart usually requires that the observed values meet the conditions of independent and same distribution, but the sample data is a time series with autocorrelation. How should the data be ...
- 11
1
vote
1
answer
64
views
IID assumption in proportion hyp test
I am asked to test a hypothesis that a manufacturing line makes p% faulty parts in a month, it's assumed that the p% is independent of the month. My approach is as simple as it gets, take a random ...
1
vote
2
answers
55
views
Can i.i.d. random variables be negative? [closed]
I know that random variables can take negative values, so can independent and identically distributed random variables be negative?
- 107
3
votes
1
answer
311
views
If a strictly stationary process is also independent, does this imply i.i.d.?
Suppose I have a time series process $\{X_t\}$ that is strictly stationary in the sense that the joint distribution of $[X_{t_1},...,X_{t_k}]$ and $[X_{t_1+a},...,X_{t_k+a}]$ are the same for any set ...
- 2,956
2
votes
1
answer
84
views
Statistical power of an experiment calculation
As titled,I had some confusion related to statistical power calculation.
The calculation is done using
...
- 31
1
vote
1
answer
97
views
Permutations of iid Random Variables [duplicate]
Suppose that $X_1, X_2, X_3$ are iid random variables. I have seen this fact many times that $$\mathbb{P}(X_1<X_2<X_3)=\frac{1}{6}$$ but I want to know that why every permutation of $X_1, X_2, ...
- 217
1
vote
2
answers
322
views
Variance of the difference of two iid sample means
Let $X_{1}, ..., X_{n}$ be random variables independent of $Y_{1}, ..., Y_{n}$, where both groups are iid with associated population means $\mu_{1}$ and $\mu_{2}$ and population variances $\sigma_{1}^{...
- 13
3
votes
1
answer
327
views
How to: Bootstrap Prediction Intervals for Regression Models with non IID noise?
Question:
Consider a regression model $Y=m(X)+\epsilon$ for which $\epsilon$ is neither independent of $x$ nor identically distributed.
How would we go about generating prediction intervals in such a ...
- 605
-2
votes
1
answer
45
views
Is part of the sequence of independent identically distributed random variables still iid? [closed]
if $x_0,x_1,...x_n$ be iid (independenta and identically distributed) random variables. The question is that the $x_0,x_3,...x_{n-1}$ is also iid?
- 107
2
votes
0
answers
38
views
A constant $c$ minus the iid random variables still iid? [duplicate]
Let $x_0,x_1,...x_n$ be iid (independenta and identically distributed) random variables. Then, $m_0,m_1,...m_n$ be defined as $c-x_0,c-x_1,....c-x_n$, where c is a constant greater than $x_i$ ($i \in \...
- 107
0
votes
1
answer
96
views
Empirical distribution function by sampling from a m.v. distribution
I have mathematically rewritten my problem as a function of multiple iid variables:
$$
f(X_1, X_2, ..., X_n),
$$
where $$X_i \in \mathcal{N}(0,1)$$
I now want to determine the empirical distribution ...
3
votes
0
answers
43
views
Are there examples of ML or stats approaches that are valid for IID data, but not exchangeable data?
Lots of supervised learning theory is motivated using the IID assumption. Do most of these methods apply equally well if data is only exchangeable, and not IID? Can you provide an example where this ...
- 416
0
votes
0
answers
443
views
What is the CDF of sum of 2 Pareto random variables?
Assume we have two independent, random variables, $X$ and $Y$ which follow the same Pareto distribution, i.e.:
$f(x) = \frac{1}{x^{2}}, x > 1$
Given that - what would be the joint probability $P(X +...
- 13
1
vote
1
answer
285
views
Central limit theorem for asymptotically i.i.d. random variables
I observe a sequence of r.v. $X_1, X_2, \dots$ where each $X_i$ is a function of the sample size $n$.
When $n \rightarrow \infty$ I have the following result: $X_1 \rightarrow^d E_1, X_2 \rightarrow^d ...
- 339
2
votes
2
answers
87
views
If $X_1,\ldots,X_n$ are continuous and IID with sample median $m$, are $1(X_1\geq m),\ldots,1(X_n\geq m)$ IID?
Suppose $X_1,\ldots,X_n$ are continuous random variables and IID with sample median $m$ for some $n\geq 2$. Is it then true that $1(X_1\geq m),\ldots,1(X_n\geq m)$ are IID?
Here the definition of $1(...
- 1,099
1
vote
0
answers
28
views
Data collection after the model is built and deployed
I have built a machine learning model which predicts whether a customer will buy a product or not. The model performs well on cross validation tests. Now, I will deploy it in production to recommend ...
- 1,272
2
votes
1
answer
418
views
Does the assumption of “independent and identically distributed” apply to the actual sample data or the sampling process?
While studying statistics, I came across the concept of “independent and identically distributed random variables” or IID.
I’m confused as to what that applies to in practice, mainly, does it apply to ...
- 23
0
votes
0
answers
28
views
$E(SN)$ for aggregate claim amount $S$, $S=X_{1}+...+X_{n}, X_{i}$ are iid [duplicate]
Consider the following model for aggregate claim amounts $S$:
$S=X_{1}+X_{2}+...+X_{N}$
where the $X_{i}$ are independent, identically distributed random variables representing individual claim ...
2
votes
1
answer
74
views
Studying extreme value r.v. $X=\max_i (c_i+X_i)$ where $c_i$ are constants and $X_i$ are i.i.d. r.v
Let
$X_1,X_2,...,X_n$ be independently and identically distributed random variables according to a distribution $F$.
There are constants: $c_1,c_2,...,c_n$.
Define a new random variable $X=\max_i(X_i+...
- 185
2
votes
0
answers
93
views
Test for distribution equality
This question touches Kolmogorov-Smirnov testing, but asks actually something different.
Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis:
$$
H_0: X_i\ ...
- 417
2
votes
1
answer
361
views
MCMC Sample should be i.i.d
I'm a bit not sure how to show that MCMC samples are i.i.d. In my opinion the trace plot should behave like white noise model because white noise model has a strong stationary properties i.e. the ...
0
votes
0
answers
403
views
If we remove half the samples from an IID dataset, is the remaining half still IID?
I am generating 10,000 pairs of X and Y such that both X and ...
- 261
0
votes
0
answers
61
views
understanding "independent" term in "independent and identically distributed" (iid) statement
I understand independent as the random variable values in a data aren't connected to each other in any way.
(q1) Is my understanding ok?
data_1: a product which has a design defect has been sold to ...
- 25
6
votes
1
answer
291
views
Bernoulli distribution with random means
Let $S = \frac{X_1 + \cdots + X_n}{n}$ where the $X_i$ are IID Bernoulli distributed with mean $p$, then $E[S] = p$ and $Var(S) = \frac{p(1-p)}{n}$.
Now consider the slightly more complex setup where $...
- 127
0
votes
1
answer
656
views
Properties of independent and identically distributed random variables [closed]
Do independent and identically distributed random variables always have the same expectation and variance?
- 1
9
votes
1
answer
437
views
References on data partitioning (cross-validation, train/val/test set construction) when data are non-IID
Consider a prediction setting in which we are interested in training a regression or classification function $f$ with inputs $X \in \mathbb{R}^k$ and target $Y$, and assessing its expected ...
- 4,384
11
votes
3
answers
3k
views
Statistical learning when observations are not iid
As far as I am concerned, statistical/machine learning algorithms always suppose that data are independent and identically distributed ($iid$).
My question is: what can we do when this assumption is ...
- 432
2
votes
2
answers
189
views
Confusion about independent and identically distributed?
Say that I wish to measure the height of male within the population (so gender=Male is the only factor I am accounting for). Say I collect 100 observations of male height from an elderly population. ...
- 21
3
votes
2
answers
221
views
Asymptotic MLE Distribution With Two Random Samples
I'm studiyng for an exam, and I found this problem which I can not managed to solve... I will be really grateful if someone can help me, thanks you.
Let $\left\{X_{1}, \ldots, X_{n}\right\} \sim^{...
- 71
1
vote
0
answers
199
views
Central limit theorem for dependent binary-related variable
Let $Y\sim N(\mu, \sigma^2)$ and given sample size $n$, we have an iid sample $\{Y_1, ..., Y_n\}$. We sample $X$ (size $n$) from Bernoulli with probability $\pi$. Denote $Z_i=X_iY_i$. Then, when $X_i=...
- 852
2
votes
1
answer
536
views
Types of noise processes and the one assumed in arima() estimation in R
Here is a time series class defining white noise incorrectly as an independent sequence of random variables.
source
Aside from the widespread mix-up of White noise and iid noise, a further ...
- 4,720
5
votes
1
answer
8k
views
What the i.i.d. assumption of the errors in linear regression implies for the response variable y?
In the linear regression model we assume that the errors $ε_i$ are independent and identically distributed (i.i.d.) random variables. I am trying to understand what this assumption implies regarding ...
- 751
1
vote
1
answer
401
views
AR/ARMA model - test residual for independence or lack of correlation
Shumway and Stoffer (2017), a great book that I highly recommend, define an AR(p) model as $x_t=\phi_1 x_{t-1}+\phi_2 x_{t-2}+...+\phi_p x_{t-p}+w_t$, where $w_t$~$wn(0,\sigma_w^2)$; $w_t$ is not ...
- 4,720
1
vote
0
answers
48
views
What is the meaning of these subscripts in iid random variables? [duplicate]
I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
0
votes
0
answers
287
views
MGF of sample mean of poisson distribution
Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$
Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$
Then, by Weak Law of Large Numbers ...
- 236
0
votes
1
answer
458
views
Clarifications on I.I.D. assumption in machine learning
In this question, it was stated that the assumption of i.i.d. for data comes in the form of
$$(X_i,y_i)∼P(X,y),∀i=1,...,N \\(X_i,y_i) \;independent\; of \;(X_j,y_j),\;∀i≠j∈{1,...,N}
$$
I am clear with ...
- 177