Questions tagged [independence]

Events (or random variables) are independent when information on some of them tells you nothing about the probability of occurrence (/ distribution) of the others. Please DO NOT use this tag for independent variable use [predictor] instead.

Filter by
Sorted by
Tagged with
38 votes
4 answers
15k views

Functions of Independent Random Variables

Is the claim that functions of independent random variables are themselves independent, true? I have seen that result often used implicitly in some proofs, for example in the proof of independence ...
user avatar
  • 18.7k
48 votes
6 answers
49k views

Why zero correlation does not necessarily imply independence

If two variables have 0 correlation, why are they not necessarily independent? Are zero correlated variables independent under special circumstances ? If possible, I am looking for an intuitive ...
user avatar
  • 6,055
108 votes
2 answers
194k views

What is covariance in plain language?

What is covariance in plain language and how is it linked to the terms dependence, correlation and variance-covariance structure with respect to repeated-measures designs?
user avatar
  • 1,767
68 votes
1 answer
173k views

Variance of product of multiple independent random variables

We know the answer for two independent variables: $$ {\rm Var}(XY) = E(X^2Y^2) − (E(XY))^2={\rm Var}(X){\rm Var}(Y)+{\rm Var}(X)(E(Y))^2+{\rm Var}(Y)(E(X))^2$$ However, if we take the product of more ...
user avatar
  • 821
42 votes
8 answers
71k views

Simple examples of uncorrelated but not independent $X$ and $Y$

Any hard-working student is a counterexample to "all students are lazy". What are some simple counterexamples to "if random variables $X$ and $Y$ are uncorrelated then they are independent"?
7 votes
3 answers
958 views

$X_i, X_j$ independent when $i≠j$, but $X_1, X_2, X_3$ dependent?

I've seen the statement: It's possible that random variables $X_i, X_j$ are independent for $i≠j$, but $X_1, X_2, X_3$ are dependent. I haven't been able to find examples of this though. Any ...
user avatar
  • 3,227
75 votes
5 answers
174k views

Covariance and independence?

I read from my textbook that $\text{cov}(X,Y)=0$ does not guarantee X and Y are independent. But if they are independent, their covariance must be 0. I could not think of any proper example yet; could ...
user avatar
  • 5,869
29 votes
2 answers
15k views

Understanding distance correlation computations

As far as I understood, distance correlation is a robust and universal way to check if there is a relation between two numeric variables. For example, if we have a set of pairs of numbers: ...
user avatar
  • 295
40 votes
3 answers
6k views

Does statistical independence mean lack of causation?

Two random variables A and B are statistically independent. That means that in the DAG of the process: $(A {\perp\!\!\!\perp} B)$ and of course $P(A|B)=P(A)$. But does that also mean that there's no ...
user avatar
11 votes
1 answer
7k views

If random variables are drawn from an identical distribution, why doesn't this guarantee they are independent?

Having read a little about exchangeability, I went back to thinking about the iid condition required for the central limit theorem. It struck me that if two random variables are drawn from an ...
user avatar
  • 121
61 votes
7 answers
44k views

How do I test that two continuous variables are independent?

Suppose I have a sample $(X_n,Y_n), n=1..N$ from the joint distribution of $X$ and $Y$. How do I test the hypothesis that $X$ and $Y$ are independent? No assumption is made on the joint or marginal ...
user avatar
  • 2,066
60 votes
13 answers
50k views

Does 10 heads in a row increase the chance of the next toss being a tail?

I assume the following is true: assuming a fair coin, getting 10 heads in a row whilst tossing a coin does not increase the chance of the next coin toss being a tail, no matter what amount of ...
user avatar
  • 601
23 votes
3 answers
10k views

Does non-zero correlation imply dependence?

We know of the fact that zero correlation does not imply independence. I am interested in whether a non-zero correlation implies dependence - i.e. if $\text{Corr}(X,Y)\ne0$ for some random variables $...
user avatar
  • 2,135
8 votes
3 answers
2k views

Testing paired frequencies for independence

I hope this isn't either far too basic or redundant. I have been looking around for guidance but so far I am still uncertain of how to proceed. My data consists of counts of a particular structure ...
user avatar
  • 4,949
7 votes
4 answers
5k views

Is " independent and identically distributed" an assumption or a fact ?

This is in the context of two random variables. A frequent assumption (e.g. of the error term in ANOVA) is of independent and identically distributed random variables. There is a question on this site ...
user avatar
6 votes
2 answers
4k views

What is the difference between using the multiplication rule or using Venn diagram subtraction for probability?

Take two problems: Andrew is 35, and the probability he will be alive in 10 years is .72. Ellen is 35, for her, .92. Assuming these are independent, what is the probability they both will be alive in ...
user avatar
  • 61
39 votes
3 answers
52k views

What is the relationship between orthogonal, correlation and independence?

I've read an article saying that when using planned contrasts to find means that are different in an one way ANOVA, constrasts should be orthogonal so that they are uncorrelated and prevent the type I ...
user avatar
20 votes
1 answer
9k views

For which distributions does uncorrelatedness imply independence?

A time-honored reminder in statistics is "uncorrelatedness does not imply independence". Usually this reminder is supplemented with the psychologically soothing (and scientifically correct) statement "...
user avatar
8 votes
2 answers
22k views

If $X$ and $Y$ are normally distributed random variables, what kind of distribution their sum follows?

I was reading this question. It is about notation but I would like to ask something regarding the sum of two normally distributed random variables. If $X$ is a normally distributed random variable ...
user avatar
  • 97
15 votes
4 answers
15k views

Can somebody illustrate how there can be dependence and zero covariance?

Can somebody illustrate, as Greg does, but in more detail, how random variables can be dependent, but have zero covariance? Greg, a poster here, gives an example using a circle here. Can somebody ...
user avatar
  • 151
30 votes
3 answers
27k views

What does "independent observations" mean?

I'm trying to understand what the assumption of independent observations means. Some definitions are: "Two events are independent if and only if $P(a \cap b) = P(a) * P(b)$." (Statistical Terms ...
user avatar
27 votes
4 answers
10k views

With categorical data, can there be clusters without the variables being related?

When trying to explain cluster analyses, it is common for people to misunderstand the process as being related to whether the variables are correlated. One way to get people past that confusion is a ...
user avatar
18 votes
4 answers
4k views

For intuition, what are some real life examples of uncorrelated but dependent random variables?

In explaining why uncorrelated does not imply independent, there are several examples that involve a bunch of random variables, but they all seem so abstract: 1 2 3 4. This answer seems to make sense....
user avatar
  • 2,196
8 votes
7 answers
3k views

Need intuition about independence of events

Suppose that we have the situation as depicted in the figure: a random experiment which has 4 outcomes $x_1, ..., x_4$ and two events $A$ and $B$. Also assume that $P(x_i)=0.25$. Now, since $P(A \cap ...
user avatar
  • 973
10 votes
4 answers
8k views

Does independence imply conditional independence?

If two or more variables A, B, C, etc. are jointly mutually independent of one another, does this imply that that they are also conditionally independent given some set of conditioning variables X, Y, ...
user avatar
  • 2,617
8 votes
1 answer
522 views

Non Linear Endogeneity

Consider the following Linear Regression Model.$$y_{it}=x_{it}\beta+\upsilon_{it}$$ where x is a scalar and $$cov(x_{it},\upsilon_{it})=0$$ It is know, however, that $cov(.)$ is a measure of ...
user avatar
  • 745
8 votes
1 answer
2k views

Estimators independence in simple linear regression

We have a simple linear model $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$ with the usual assumptions, $i \in \{1, \cdots, n\}$. Let $\hat{\beta_1}$ and $\hat{\sigma}^2$ be the least-square estimators for $\...
user avatar
  • 295
4 votes
2 answers
6k views

Covariance of products of dependent random variables

I have four random variables, A, B, C, D, with known mean and variance. As well: Cov(A,B) is known and non-zero Cov(C,D) is known and non-zero A and C are independent A and D are independent B ...
user avatar
  • 143
3 votes
2 answers
273 views

Independence of events in real-life data

Most of statistical methods (if not all) rely on independence of events. How do we know that this assumption is valid in real-life problems like clinical trials or web crawling? What might be the ...
user avatar
  • 161
1 vote
1 answer
147 views

Sum of indepedent random variables and a constant

Let $X_1$ and $X_2$ be independent Normal random variables with mean $\mu_1$ and $\mu_2$, and variances $\sigma_1$ and $\sigma_2$. Let $Y = X_2-X_1 + c$, where $c$ is a constant. For notational ...
user avatar
24 votes
1 answer
2k views

Plain language meaning of "dependent" and "independent" tests in the multiple comparisons literature?

In both the family-wise error rate (FWER) and false discovery rate (FDR) literature, particular methods of controlling FWER or FDR are said to be appropriate to dependent or independent tests. For ...
user avatar
  • 26.7k
12 votes
2 answers
33k views

What is the demonstration of the variance of the difference of two dependent variables?

I know that the variance of the difference of two independent variables is the sum of variances, and I can prove it. I want to know where the covariance goes in the other case.
user avatar
  • 129
12 votes
2 answers
12k views

Alternatives for chi-squared test for independence for tables more than 2 x 2

What are some alternatives to the chi-squared test for categorical variables with tables larger than 2 x 2 and cells with a count less than 5, if I don't want to merge classes?
user avatar
  • 133
18 votes
5 answers
1k views

Does this quantity related to independence have a name?

Obviously events A and B are independent iff Pr$(A\cap B)$ = Pr$(A)$Pr$(B)$. Let's define a related quantity Q: $Q\equiv\frac{\mathrm{Pr}(A\cap B)}{\mathrm{Pr}(A)\mathrm{Pr}(B)}$ So A and B are ...
user avatar
8 votes
3 answers
290 views

Why is dependence a problem?

I'm interested in why dependent observations are a problem in statistics. Let's say you want to know if there is a difference in mean exam scores between two schools. You collect 50 observations in ...
user avatar
  • 12.7k
4 votes
2 answers
3k views

Conditional independence isn't guaranteed when specifying the marginal distributions?

It was mentioned very briefly in a lecture related to graphical models that two random variables $X_3$ and $X_4$ are both dependent on $X_2$. But even when conditioned on $X_2$, the two variables $...
user avatar
7 votes
6 answers
4k views

When two events $A$ and $B$ have no result in common

I and my friends just had a little discussion whether the events are independent or dependent if they have no outcome in common. I thought that they have to be independent. When two events are ...
user avatar
  • 211
1 vote
1 answer
133 views

Question about correlations and independence

My professor was discussing correlation and when it implies independence. It was fairly clear to me that, if $X$ and $Y$ are independent, then their correlation is zero. The reverse direction is less ...
user avatar
  • 135
42 votes
10 answers
23k views

Are your chances of dying in a plane crash reduced if you fly direct?

I recently had a disagreement with a friend about minimizing the chance of dying in a plane due to a crash. This is a rudimentary statistics question. He stated that he prefers to fly direct to a ...
user avatar
  • 531
23 votes
2 answers
20k views

Test for IID sampling

How would you test or check that sampling is IID (Independent and Identically Distributed)? Note that I do not mean Gaussian and Identically Distributed, just IID. And idea that comes to my mind is ...
user avatar
  • 13.5k
24 votes
2 answers
25k views

Data augmentation techniques for general datasets?

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, ...
user avatar
  • 909
7 votes
1 answer
6k views

What is implied by i.i.d.?

It's common to see statements that data must be i.i.d. But if data in a time series are independent, aren't they just noise? At one point I thought, they're only referring to the errors, i.e. the ...
user avatar
13 votes
2 answers
16k views

Test of independence vs test of homogeneity

I am teaching a basic statistics course and today I will cover the chi-squared test of independence for two categories and the test for homogeneity. These two scenarios are conceptually different, but ...
user avatar
  • 13.6k
6 votes
3 answers
411 views

Under what additional conditions does independence follow from zero correlation?

Under what conditions does the below statement hold: X and Y are uncorrelated if and only if X and Y are independent. I totally understand that this statement does not always hold, but I would ...
user avatar
  • 125
25 votes
10 answers
9k views

Confused about independent probabilities. If a fair coin is flipped 5 times, P(HHHHH) = 0.03125, but P(H | HHHH) = 0.5

I'm confused about how to reconcile the probability of independent events not having anything to do with prior history, but sequences of events do (seemingly) take into account prior history. This ...
user avatar
  • 395
8 votes
2 answers
10k views

What's the difference between "mean independent" and independent?

As stated in the Econometrics textbook (Introductory Econometrics by Wooldbridge): When $E(u|x)=E(u)$ holds, we say that $u$ is mean independent of $x$. Why can't we simply say that $u$ is ...
user avatar
  • 609
13 votes
1 answer
16k views

Variance of annual return based on variance of monthly return

I'm trying to understand the whole variance/std error thing of a time series of financial returns, and I think I'm stuck. I have a series of monthly stock return data (let's call it $X$), which has ...
user avatar
  • 413
6 votes
2 answers
2k views

10% rule for sample sizes

In an introductory stats book by Nicole Radziwill "Statistics (the easier way) with R", an assumption used for nearly every statistical test (e.g.t-tets, anova, etc) is that the sample size ...
user avatar
5 votes
3 answers
2k views

Are observations independent in bootstrapped resamples?

The bootstrap is often used for nonparametric inference. However, in some cases, it is useful to bootstrap and then conduct parametric tests within each resample (optionally, see References, but this ...
user avatar
  • 3,501
11 votes
2 answers
2k views

Is the sum of two variables independent of a third variable, if they are so on their own?

Given 3 random variables $X_1$, $X_2$ and $Y$. $Y$ and $X_1$ are independent. $Y$ and $X_2$ are independent. Intuitively I would assume that $Y$ and $X_1+X_2$ are independent. Is this the case, and ...
user avatar
  • 467