Questions tagged [iid]

iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.

Filter by
Sorted by
Tagged with
3 votes
0 answers
21 views

Are there examples of ML or stats approaches that are valid for IID data, but not exchangeable data?

Lots of supervised learning theory is motivated using the IID assumption. Do most of these methods apply equally well if data is only exchangeable, and not IID? Can you provide an example where this ...
0 votes
0 answers
17 views

What panel data assumptions hold true when the underlying process is pooled cross sectional?

Most econometric textbooks convey that panel data contains both an entity and a time component, and that the entity can be individuals, firms, counties etc. The assumption we need to deal with in ...
  • 1
0 votes
0 answers
55 views

What is the CDF of sum of 2 Pareto random variables?

Assume we have two independent, random variables, $X$ and $Y$ which follow the same Pareto distribution, i.e.: $f(x) = \frac{1}{x^{2}}, x > 1$ Given that - what would be the joint probability $P(X +...
1 vote
1 answer
118 views

Central limit theorem for asymptotically i.i.d. random variables

I observe a sequence of r.v. $X_1, X_2, \dots$ where each $X_i$ is a function of the sample size $n$. When $n \rightarrow \infty$ I have the following result: $X_1 \rightarrow^d E_1, X_2 \rightarrow^d ...
  • 63
0 votes
0 answers
28 views

Steiger's Z test, independent samples required?

I have some samples and for each one, I have 3 measures : x, y and z. I want to know if the correlation between x and y and the correlation between y and z are significantly different. I will use ...
2 votes
2 answers
67 views

If $X_1,\ldots,X_n$ are continuous and IID with sample median $m$, are $1(X_1\geq m),\ldots,1(X_n\geq m)$ IID?

Suppose $X_1,\ldots,X_n$ are continuous random variables and IID with sample median $m$ for some $n\geq 2$. Is it then true that $1(X_1\geq m),\ldots,1(X_n\geq m)$ are IID? Here the definition of $1(...
  • 1,068
1 vote
0 answers
24 views

Data collection after the model is built and deployed

I have built a machine learning model which predicts whether a customer will buy a product or not. The model performs well on cross validation tests. Now, I will deploy it in production to recommend ...
  • 1,023
2 votes
1 answer
38 views

Does the assumption of “independent and identically distributed” apply to the actual sample data or the sampling process?

While studying statistics, I came across the concept of “independent and identically distributed random variables” or IID. I’m confused as to what that applies to in practice, mainly, does it apply to ...
0 votes
0 answers
25 views

$E(SN)$ for aggregate claim amount $S$, $S=X_{1}+...+X_{n}, X_{i}$ are iid [duplicate]

Consider the following model for aggregate claim amounts $S$: $S=X_{1}+X_{2}+...+X_{N}$ where the $X_{i}$ are independent, identically distributed random variables representing individual claim ...
1 vote
1 answer
61 views

Studying extreme value r.v. $X=\max_i (c_i+X_i)$ where $c_i$ are constants and $X_i$ are i.i.d. r.v

Let $X_1,X_2,...,X_n$ be independently and identically distributed random variables according to a distribution $F$. There are constants: $c_1,c_2,...,c_n$. Define a new random variable $X=\max_i(X_i+...
  • 143
2 votes
0 answers
44 views

Test for distribution equality

This question touches Kolmogorov-Smirnov testing, but asks actually something different. Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis: $$ H_0: X_i\ ...
2 votes
1 answer
121 views

MCMC Sample should be i.i.d

I'm a bit not sure how to show that MCMC samples are i.i.d. In my opinion the trace plot should behave like white noise model because white noise model has a strong stationary properties i.e. the ...
0 votes
0 answers
168 views

If we remove half the samples from an IID dataset, is the remaining half still IID?

I am generating 10,000 pairs of X and Y such that both X and ...
0 votes
0 answers
37 views

understanding "independent" term in "independent and identically distributed" (iid) statement

I understand independent as the random variable values in a data aren't connected to each other in any way. (q1) Is my understanding ok? data_1: a product which has a design defect has been sold to ...
  • 25
6 votes
1 answer
219 views

Bernoulli distribution with random means

Let $S = \frac{X_1 + \cdots + X_n}{n}$ where the $X_i$ are IID Bernoulli distributed with mean $p$, then $E[S] = p$ and $Var(S) = \frac{p(1-p)}{n}$. Now consider the slightly more complex setup where $...
  • 127
0 votes
1 answer
131 views

Properties of independent and identically distributed random variables [closed]

Do independent and identically distributed random variables always have the same expectation and variance?
6 votes
1 answer
174 views

References on data partitioning (cross-validation, train/val/test set construction) when data are non-IID

Consider a prediction setting in which we are interested in training a regression or classification function $f$ with inputs $X \in \mathbb{R}^k$ and target $Y$, and assessing its expected ...
  • 3,850
9 votes
3 answers
825 views

Statistical learning when observations are not iid

As far as I am concerned, statistical/machine learning algorithms always suppose that data are independent and identically distributed ($iid$). My question is: what can we do when this assumption is ...
1 vote
2 answers
80 views

Confusion about independent and identically distributed?

Say that I wish to measure the height of male within the population (so gender=Male is the only factor I am accounting for). Say I collect 100 observations of male height from an elderly population. ...
  • 11
2 votes
2 answers
104 views

Asymptotic MLE Distribution With Two Random Samples

I'm studiyng for an exam, and I found this problem which I can not managed to solve... I will be really grateful if someone can help me, thanks you. Let $\left\{X_{1}, \ldots, X_{n}\right\} \sim^{...
1 vote
0 answers
66 views

Central limit theorem for dependent binary-related variable

Let $Y\sim N(\mu, \sigma^2)$ and given sample size $n$, we have an iid sample $\{Y_1, ..., Y_n\}$. We sample $X$ (size $n$) from Bernoulli with probability $\pi$. Denote $Z_i=X_iY_i$. Then, when $X_i=...
  • 799
1 vote
1 answer
197 views

Types of noise processes and the one assumed in arima() estimation in R

Here is a time series class defining white noise incorrectly as an independent sequence of random variables. source Aside from the widespread mix-up of White noise and iid noise, a further ...
3 votes
1 answer
2k views

What the i.i.d. assumption of the errors in linear regression implies for the response variable y?

In the linear regression model we assume that the errors $ε_i$ are independent and identically distributed (i.i.d.) random variables. I am trying to understand what this assumption implies regarding ...
  • 691
0 votes
1 answer
127 views

AR/ARMA model - test residual for independence or lack of correlation

Shumway and Stoffer (2017), a great book that I highly recommend, define an AR(p) model as $x_t=\phi_1 x_{t-1}+\phi_2 x_{t-2}+...+\phi_p x_{t-p}+w_t$, where $w_t$~$wn(0,\sigma_w^2)$; $w_t$ is not ...
1 vote
0 answers
47 views

What is the meaning of these subscripts in iid random variables? [duplicate]

I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
0 votes
0 answers
22 views

How to calculate Maximum Likelihood Estimator

I have samples of a noisy real vector with constant phase y¯(2) = a¯ . e^jθ + w where θ is a real scalar, and the entries of w¯ are complex normal i.i.d, where W i,real, W i,image ∼ N(0,σ^2) for i = 1,...
  • 121
0 votes
0 answers
96 views

MGF of sample mean of poisson distribution

Let $X_1,X_2,\dots,X_n\stackrel{iid}{\sim}Poiss(\lambda)$ Let mgf of $X_1$ is given by $M_X(t)=e^{\lambda(e^t-1)}$ and let $\bar{X_n}=\frac{1}{n}(X_1+X_2+\dots+X_n)$ Then, by Weak Law of Large Numbers ...
  • 187
0 votes
1 answer
63 views

Clarifications on I.I.D. assumption in machine learning

In this question, it was stated that the assumption of i.i.d. for data comes in the form of $$(X_i,y_i)∼P(X,y),∀i=1,...,N \\(X_i,y_i) \;independent\; of \;(X_j,y_j),\;∀i≠j∈{1,...,N} $$ I am clear with ...
  • 177
0 votes
1 answer
50 views

Confusion in definition of independent and identically distributed random variables [duplicate]

From what I learnt, a random variable is a function which assigns real values to outcome space, and the probability distribution is a function that assigns probability to different values produced by ...
0 votes
0 answers
54 views

Obtaining better estimates when you know that a set of input variables are independently and identically distributed

Suppose I have a family of random variables $$X_i \sim SomeDistribution_i, \ \ i = 1,..., n$$ and I know how to sample $SomeDistribution_i$ for any $i$. Suppose I also define a random variable $Y = f(...
2 votes
0 answers
50 views

ISI PCB NC$9$ Limiting Distribution of Bernoulli to Poisson

Let $X_i\sim (i.i.d.)$, Bernoulli($\frac{\lambda}{n}$), $n\ge \lambda\ge 0$. $Y_i\sim (i.i.d.)$, Poisson($\frac{\lambda}{n}$). $\{X_i\}$ and $\{Y_i\}$ are independent. Define $T_n=\sum_{i=1}^{n^2}X_i$...
  • 121
3 votes
0 answers
84 views

Example of independent observations

I came across an interesting (to me, at least) question. I want to predict the probability that a given boat will win a race. The dataset is something like: Y: winner (0-1) X1: size of the sail X2: ...
  • 31
1 vote
0 answers
34 views

Trying to understand iid [closed]

Suppose that I have a sequence of random variables $\{X_i,Y_{it}\}_{i=1,t=1}^{N,T}$. I want to assume i.i.d with this sequence of random variables. So I wonder it is correct to assume $\{X_i,Y_{it}\}...
3 votes
1 answer
71 views

Why is MAP and ML widely accepted? [closed]

(ML as in Maximum Likelihood and MAP as in Maximum A-posteriori) I'm going trough a course book on my own, and without really having peers to talk to I'm turning to stack exchange with these rather ...
  • 43
0 votes
1 answer
189 views

Non IID data and SVM Classifier

I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
3 votes
1 answer
45 views

Difference of independent random variables that is not unimodal

This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
  • 526
2 votes
0 answers
226 views

Posterior distribution of two i.i.d. uniform r.v. given their difference with graphical intuition

I have two i.i.d. random variables, $\theta_1$ and $\theta_2$ which are uniformly distributed on the unit square. I need to compute the joint posterior distribution of these two variables, given their ...
  • 31
0 votes
1 answer
71 views

Probability with two IID random variables

Assume $X$ and $Y$ are two IID random variables with infinite support, I am interested in $P(X>a , X>Y+b)$ where $a$ and $b$ are two constants. Is there any family of distribution for X and Y ...
  • 83
3 votes
1 answer
106 views

Central limit theorem for the function of an iid random variable

Given an iid random variable $X$, instead of the distribution $\sqrt{n}(n^{-1}\sum{X_{i}}-E[X])$ which is the result that the central limit theorem provides , I am interested in the distribution of $\...
  • 134
1 vote
0 answers
20 views

What happens if I use OLS in a multiple (hedonic) regression where prices are systematically missing?

I am using a hedonic regression of (log) housing prices on a set of price-determining characteristics. I then use the estimated coefficients to estimate housing prices for observations not in my ...
  • 1,168
1 vote
1 answer
137 views

Definition of independent events in probability theory (Wasserman)

In Wasserman's "All of Statistics" p.26 he gives an example of an "independent event" as "flipping a fair coin twice", where the first flip has no effect on the second ...
1 vote
0 answers
114 views

Are stationary markov chains iid random variables?

Let $\{X_t\}_{t=1}^{\infty}$ be a Markov Chain. An initial marginal distribution $\pi^T$ for a markov chain is a stationary distribution if $\pi^TP = \pi^T$. My understanding of this is that if the ...
  • 33
1 vote
0 answers
72 views

Machine Learning IID [duplicate]

I am new in ML so excuse me if this is a bit basic. I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent). I don't exactly ...
  • 43
1 vote
1 answer
238 views

i.i.d assumption: formal definition vs. intuition [duplicate]

Intuition In ML, as I constantly run into the i.i.d assumption for datasets, I have an intuition of what this assumption really means. So if I'm not mistaken: "independent" means that ...
  • 135
0 votes
1 answer
64 views

Is there any need of random sampling for IID Data?

I understand that random sampling is required for the purpose of creating an unbiased sample with the same characteristics as the population. I am confused about whether random sampling is required ...
0 votes
1 answer
60 views

Iid random variables with infinite variance are unbounded

While preparing for an exam I've stumbled upon an exercise I have no idea how to approach: $X_1, \dots, X_n$ are iid random variables with $E(X_1) = 0$ and $V(X_1)=\infty$ Show that $\mathbb{P}(\{ |...
2 votes
1 answer
100 views

Confounding variables VS i.i.d assumption

I made up an example so as to illustrate my question with some more context. Say there are two national parks, and a ranger is interested in finding out how the number of rabbits (Y) varies with the ...
  • 23
3 votes
1 answer
113 views

What is the expectation of $\left\langle (n \bar{y})^4 \right\rangle$, if $y_i \sim \mathcal{N}(\mu,\sigma^2)$?

Let $y_i \sim \mathcal{N}(\mu,\sigma^2), \; i = 1,\ldots,n$ and $\bar{y} = \frac{1}{n} \sum_{i=1}^n y_i$, such that $n \bar{y} = y_1 + \ldots + y_n$. Then, we want to know what the expectation of $(n \...
0 votes
0 answers
32 views

When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? [duplicate]

When can we reasonably assume a sequence of r.v.'s is IID in real life scenarios? My question is based off the following example from Wasserman's All of Statistics: Suppose we test a prediction ...
1 vote
2 answers
1k views

Summation of i.i.d. Normal Random Variables

Assuming I have $$X_1,X_2,...,X_{100}\sim N(1,4)$$ and $$Y_1,Y_2,...,Y_{20}\sim N(2,9)$$ where all $X$ are iid, all $Y$ are iid. Then should $$\text{var}(X_1+X_2+\ldots+X_{100}+Y_1+\ldots + Y_{20}) = ...

1
2 3 4 5