Questions tagged [iid]
iid is an acronym for independent and identically distributed. Many statistical methods assume that the data are iid; that is, that each observation comes from the same distribution and is independent of other observations.
176
questions
0
votes
0answers
5 views
Determine Expected Value and Bias of Gamma Distribution [closed]
Let $X_1,β¦, X_n$ be a sample of $iid$ $Gamma(4, π)$, and let $T = \frac{1}{4} \overline{X}$ be an
estimator of $π$. Determine:
a) $E(T)$
b) $Bias(T;π)$
c) $MSE(T;π)$
d) whether $T$ is an MSE-...
0
votes
0answers
11 views
Estimators that relate model accuracy to data distributions
Consider a set of dependent variables $D$, set of independent variables $I$ and $O_1, O_2,\ldots,O_N$ observations of these variables. The dataset is short and wide, $|D| = 3*N$.
For each $j \in I$, ...
0
votes
0answers
20 views
Does permutation test require iid?
Suppose I have a treatment group X and independent from control group Y.
The data set x1,x2..., y1,y2... Let's say I want to test the mean of the difference
If the randomization unit is a session ...
6
votes
1answer
47 views
Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed
In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are ...
1
vote
2answers
41 views
Independent and identically distributed data (images)?
If it is said that the data must be independent and identically distributed, and the data are images, then what exactly does it mean for images to be "independent and identically distributed"...
-1
votes
1answer
78 views
Does a version of the Delta Method exist for non-i.i.d. sequences?
I have a sequence of random variables that are non-independent, but usually identically distributed. I am wondering if a version of the Delta Method exists under the case when I only have that the ...
1
vote
1answer
33 views
Expected vallue calculation of i.i.d. random variables
Suppose $X_1,X_2,\ldots,X_n$ are a sequence of i.i.d. random variables with mean $\mu$ and variance $\sigma^2$.
Define the sample mean $\bar{X} := \frac{1}{n} \sum_{i=1}^{n} X_i$, which we know is an ...
0
votes
0answers
39 views
How do you compare samples if they are not Independent and Identical Distributed (IID)?
I have 2 questions.
Nr. 1: I have a simulation model running for 1 year (after the warm-up period) that simulates the patients coming into a clinic for their treatment (set up daily) (a patient may ...
2
votes
1answer
44 views
What does i.i.d. mean for multivariate case?
When we say a random variable is i.i.d., it's often used to describe the dependency between the observations of that random variable, which I call the row dimension, indexed by time if it's a time ...
2
votes
1answer
38 views
Is there a central limit theorem for random variables with a bounded interval? [duplicate]
Is there any theorem which states the asymptotic distribution for the sample mean when the samples are drawn from a random variable which has a bounded interval?
1
vote
1answer
29 views
What is a random process with “stationary independent increments”?
I'm looking at a Solved Problem in "Schaum's Outline: Probability, Random Variables, and Random Processes", specifically Problem 5.21. In this problem it states:
Let $\{X(t), t \ge 0\}$ be ...
1
vote
1answer
42 views
How are cross validation and i.i.d. assumption of of a dataset related?
Is it necessary for the observations of the data set to be IID in order to use cross-validation on it? If so, why ? Could you explain in the context of a classification using decision tree.
0
votes
0answers
10 views
Apply panel data techniques to a panel where observation can be assumed to be independent?
I'm working with a data-set and am unsure if I should apply specific panel data techniques to it or not.
The data consists of panel data for municipalities in the Philippines and damage cause to rice ...
5
votes
1answer
91 views
Is a sample i.i.d or is a collection of random variables i.i.d.?
Basic terminology question.
I hear βlet the sample be i.i.d.β and βlet these random variables be i.i.d.β being used interchangeably.
Even Wikipedia uses both:
A collection of random variables is ...
0
votes
0answers
9 views
Does the mean preserving spread of a distribution constitute a mean preserving spread of the joint distribution of two iid draws from it?
Let random vectors $X_1, X_2 \sim F, \;i.i.d, X_1, X_2 \in X $. Now replace $F$ with its mean-preserving spread (MPS), say $G$. My question is, does that constitute an MPS of the joint distribution of ...
2
votes
2answers
36 views
Accept-reject and subsets of iid samples
I have some confusion about subsets of iid samples being distributed as the original sample.
As an illustration, consider the accept-reject algorithm to produce iid samples from a pdf $f(x)$. We draw,...
1
vote
0answers
18 views
How to derive this MAE error bound on the central limit theorem?
Is this derived from Chebyshev's inequality or a tail bound theorem? If not, how was it derived?
Does this require the existence of the third moment?
Does this bound suggest the normal approximation ...
0
votes
0answers
46 views
I.I.D. for the layperson? [duplicate]
Question: For the layman, what does it mean for data (say $n$ samples covering $m$ variables) to be identically distributed, and how is it practically achieved when conducting machine learning?
So ...
0
votes
0answers
14 views
Is a non-iid process necessarily a Markov Process?
This may be a silly question, but I've been wondering if a a random variable is said to follow a non-iid process, does that necessarily imply that it follows a Markov process?
1
vote
1answer
124 views
When using a gaussian link in GLM, what are the assumptions?
In R, when I am fitting a model glm(y~x, family = gaussian(link="log")), do I assume that $Y \stackrel{iid}\sim N(\mu, \sigma^2)$ or do I assume that $Y \stackrel{...
0
votes
0answers
48 views
How to compute ESS (Effective Sample Size)?
I implemented the ESS calculation according to this manual like this:
...
1
vote
0answers
43 views
What if my samples are not IIDs? [closed]
I'm testing my system and getting some results over time (Queues size, wating time, ecc.). When I plot the correlograms they exhibit a high degree of autocorrelation, even for high lags. Now, what are ...
0
votes
0answers
70 views
What exactly is p(x,y) in the context of iid assumption in machine learning?
In machine learning iid assumption means that examples in the dataset are independent and drawn from the same probability distribution (i.e., identically distributed).
Here, the probability ...
12
votes
3answers
1k views
Realistically, does the i.i.d. assumption hold for the vast majority of supervised learning tasks?
The i.i.d. assumption states:
We are given a data set, $\{(x_i,y_i)\}_{i = 1, \ldots, n}$, each data $(x_i,y_i)$ is generated in an independent and identically distributed fashion.
To me, ...
1
vote
0answers
39 views
What problems happen when $L(\theta|x_1,x_2,…,x_n)$ be a function of differently distributed random variables?
Given $L(\theta|x_1,x_2,...,x_n)$, may $x_1,x_2,...,x_n$ not all be identically distributed? What problems happen in the not identically distributed case compared to the iid case?
1
vote
0answers
22 views
Shouldnβt we say independent given the distribution?
In statistics we often deal with iid random variables: independent identically distributed. But if we donβt know the distribution (say we still know the support is {0, 1}), and we get a sample x1, say ...
2
votes
1answer
222 views
Why is the MLE for variance in single linear regression biased? [duplicate]
I understand that the Maximum Likelihood Estimator for variance, in general, is biased (the average calculated from the sample itself reduces the degree of freedom by 1 e.t.c):
...
1
vote
0answers
283 views
Probability that one positive random variable is greater than another independent and identically distributed positive random variable
Suppose I have the following random variables $X_{0} \sim f(x)$ and $X_{1} \sim f(x)$ are independent. I want to know the probability that $X_{0} > X_{1}$. So I think I want to find:
$$ P(X_{0}>...
2
votes
1answer
339 views
Probability of a machine failing when components fail independently
Our machine is made out of 3634 units of component A which has failure rate of 10%, 1656 units of component B with failure rate of 35% and 3368 units of component C with failure rate of 55%. Assuming ...
3
votes
1answer
405 views
UCB Exploration in Reinforcement Learning
I have two questions regarding the upper confidence bounds (UCB) exploration in reinforcement learning:
UCB exploration is derived from Hoeffding's inequality which assumes that the reward is bounded ...
0
votes
0answers
60 views
Bootstrapping regression coefficient for time series
One of the fundamental assumptions of bootstrap is that the samples are independent and identically distributed (i.i.d). This is the reason why it is difficult to bootstrap time-series because the ...
1
vote
2answers
372 views
Random Variable with IID always Gaussian?
Is there a case when we assume a random variable $\epsilon$ to be IID and assume its distribution is not gaussian?
0
votes
0answers
35 views
How to check Gauß-Markov theorem after OLS estimation?
I have estimate a simple bivariate regression with OLS and want to proof if the estimator is unbiased.
I found that the GauΓ-Markov theorem consists of 4 Assumptions and the first 3 can be rewrite as ...
2
votes
1answer
123 views
Expectation Value of a Product of Many IID variables
First of all, I apologize for not being rigorous, but I am not a statistitian by background.
Imagine you have $N$ i.i.d. positive random variables $X_1...X_N$ and you are trying to compute a ...
3
votes
4answers
311 views
Check if residuals are IID (timeseries)
How can I check in R after decomposing a time series if my residual component is IID noise?
Would this be the best way (to use the autocorrelation function) and check for 0 correlation on all lags > ...
4
votes
0answers
208 views
What is the fundamental assumption for using resampling methods?
Suppose that I observe a set of non-i.i.d. data (time series) $\mathcal{L} = \left\lbrace (y_{t}, x_{t}) \right\rbrace_{t=1}^{T}$ with $x_{t} = (x_{t1},\ldots,x_{tP})$ a real valued vector of $P$ ...
1
vote
0answers
28 views
When does $E[f(X_i)]=E[f(X_j)], i\neq j$?
Suppose we have random variables $X_1, \dots, X_N$, with joint probability distribution $F_{X_1,\dots,X_N}$. Under what conditions does the following equality holds?
$$E[f(X_i)]=E[f(X_j)],\ \ i\neq ...
6
votes
2answers
494 views
Expectation of inverse of sum of positive iid variables
Let $(X_i)_{i}$ be a sequence of iid positive variables of mean 1 and variance $\sigma^2$. Let $\bar{X}_n = \frac{\sum_{i=1}^n X_i}{n}$.
My question is: Can we can bound $\mathbb{E}(1/\bar{X}_n)$ as ...
0
votes
1answer
309 views
Are i.i.d. random variables always associated with a distribution function? [closed]
It is known that independent and identically distributed (i.i.d.) random variables are mutually independent and each random variable has the same probability distribution as the others.
However, is ...
1
vote
1answer
72 views
Implications of i.i.d. sample
I have the following question:
I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as
$\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
48 views
Are feature values random variable realizations, or random variables themselves?
I'm new to ML/stats so got confused with what I supposed was simple notation.
For simplicity, say I have a data set with just one column:
From probabilistic perspective, I had understood that X ...
6
votes
1answer
83 views
Is MLE intrinsically connected to logs?
My mathematical exploration led me the following claim:
Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). Itβs not correct to say log shows up simply to make ...
1
vote
0answers
37 views
Functional data analysis references
I am working on a inferential framework for functional data. I have categorical independent variables and a functional response variables. Different implementations of functional linear models are ...
3
votes
1answer
224 views
Does joint IID imply marginal IID?
Suppose you have a 2 dimension random vector denoted $(X,Y)$ that is Independent and Identically Distributed (IID) for a sequence of draws $((X_1,Y_1),(X_2,Y_2),...,(X_n,Y_n))$. Does this imply that ...
8
votes
3answers
170 views
Calculate the variance of $\sum\limits_{i=1}^{n-1} \sum\limits_{j=i+1}^n S(X_i - X_j)$ for $X_1,\ldots,X_n$ i.i.d. random variables
In p.88 of Wand & Jones (1995), they asked to show the following result.
Let $X_1,\ldots,X_n$ be a set of i.i.d. random variables and define $$U=2n^{-2}\sum_{i=1}^{n-1} \sum_{j=i+1}^n S(X_i - ...
5
votes
2answers
245 views
Identically distributed vs P(X > Y) = P(Y > X)
I've two related propositions which seem correct intuitively, but I struggle to prove them properly.
Question 1
Prove or disprove: If $X$ and $Y$ are independent and have identical marginal ...
1
vote
0answers
28 views
What final test to apply to a permutation test (by compression) algorithm?
I'm following in the steps of §5.1 Permutation Testing, NIST Special Publication 800-90B, Recommendation for the Entropy Sources Used for Random Bit Generation.
I'm developing my test for ...
0
votes
1answer
112 views
Likelihood when points aren't i.i.d
If we assume that we have a set of N data points given as $\textbf{X}$ and corresponding targets vectors $\textbf{T}$, where both represents matrices in this case. For an i.i.d we could write the ...
1
vote
0answers
136 views
Estimate unknown sum of iid random variables
Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed discrete random variables with common mass function $f_X(x)$ defined for when $x \in \{0,1,\dots,N\}$ and $N$ is known.
...
3
votes
1answer
241 views
Are RNNs inherently flawed? Supervised Learning assumes IID data but sequential data is not IID
From what I understand, Supervised Learning operates under the assumption that the data is I.I.D. It seems to me that the training procedure for RNNs is flawed. We receive observations in a sequential ...
