4
$\begingroup$

I am confused by the terminology because the term "independent samples" is usually used as a synonym for "unpaired samples". Does this fact mean that unpaired samples are always generated by independent random variables (assuming we have a probability model for our data)?

In other words, let's assume that we have a probability model for our data, that is we have two random variables: $X \sim F_X$, $Y \sim F_Y$. These two random variables correspond to two populations. Next, we consider two i.i.d. random samples $(X_1, \ldots, X_n) \overset{\text{iid}}{\sim} F_X$, $(Y_1, \ldots, Y_m) \overset{\text{iid}}{\sim} F_Y$ where there is no one element of $(X_1,\ldots,X_n)$ such that we can "match" it with some element of $(Y_1, \ldots, Y_m)$. We call such samples unpaired samples (of random variables).
Is it true that $X_i$ and $Y_j$ are independent random variables, $\forall i,j$? Or, equivalently, is it true that $X$ and $Y$ are independent random variables?


P.S. It is well known that paired samples (often called dependent samples) can be generated either by dependent or by independent jointly distributed random variables $X$ and $Y$ (see examples 1 and 2 below). But my question above is about unpaired samples.

Example 1. We have $n$ random people and measure the same characteristic (for example, body weight) in two different moments of time, this will give us $n$ pairs of numbers $(x_1,y_1), \ldots, (x_n,y_n)$. Here we can treat these numbers as realization of two paired samples (of random variables) $(X_1,\ldots,X_n) \overset{\text{iid}}{\sim} F_X$ and $(Y_1,\ldots,Y_n)\overset{\text{iid}}{\sim} F_Y$, which were generated by two dependent jointly distributed random variables $X$ and $Y$.
Example 2. We have $n$ random people and measure two totally different, unrelated characteristics (for example, year of birth and gender), this gives us $n$ pairs of numbers $(x_1,y_1),…,(x_n,y_n)$. Here we can treat these numbers as realization of two paired samples (of random variables) $(X_1,\ldots,X_n) \overset{\text{iid}}{\sim} F_X$ and $(Y_1,\ldots,Y_n) \overset{\text{iid}}{\sim} F_Y$, which were generated by two independent jointly distributed random variables $X$ and $Y$.

If we have any doubts, we can use the chi-squared test of independence to find out if paired samples $(X_1,\ldots,X_n)$ and $(Y_1,\ldots,Y_n)$ are generated by dependent or independent jointly distributed random variables $X$ and $Y$.

$\endgroup$

2 Answers 2

3
+50
$\begingroup$

I am confused by the terminology because the term "independent samples" is usually used as a synonym for "unpaired samples". Does this fact mean that unpaired samples are always generated by independent random variables (assuming we have a probability model for our data)?

No, 'unpaired data' is not always independent.

The answer below gives is first an interpretation of how 'unpaired' relates to independence. After that, it gives two examples of how two samples can still be dependent, even when there is no pairing.

Unpairing data

Yes, you do have that a set of pairs of data lose their dependency when you switch the labeling.

The example below shows what happens when we remove the pairing of two correlated variables.

See that point at the top in the left graph, around $x,y = 2,2.5$, if you have unpaired data, then the x-coordinate that is matched with this $y = 2.5$ can suddenly be anything from the distribution of $x$ values.

intuitive graph


An unpaired way of dependency

Independence between samples occurs if the outcomes of the two variables are unrelated. So if the probability distribution $f_Y(y)$ is not dependent on the $X_i$ and vice versa if the probability distribution $f_X(x)$ is not dependent on the $Y_i$.

Gathering samples pairwise, such that variables might have some relation, is one practical setting where variables might have a dependency. Due to being sampled within the same unit (e.g. same time, person, place, etc.) the probability distribution of the one element in the pair can be depending on the value of the other element in the pair.

But, there are other ways in which the sample $X$ could influences the density $f_Y(y)$, yet not in terms of a pairwise relationship (or generalized multiple comparisons beyond the number pair/two).

For instance, the parameters in $f_Y(y)$ could depend on $\sum X_i$.

Example consider two i.i.d. random samples of size $n$

$$\begin{array}{rcl} (X_1, \ldots, X_n) & \overset{\text{iid}}{\sim} & N(0, 1) \\ (Y_1, \ldots, Y_n) & \overset{\text{iid}}{\sim} & N(\mu,\sigma^2) \end{array}\\ \text{with $\mu = \frac{1}{n}\sum_{i=1}^{n}{X_i}$ and $\sigma^2 = \frac{1}{n}\sum_{i=1}^{n}{(X_i-\mu)^2}$} $$

Non-explicit paired data but related

It might also be that you have two variables that are not explicitly paired, and are not stated as 'paired data', but are dependent when they are combined together based on additional metadata. For example recordings of cloudiness and recordings of rainfall from two different datasets can be 'paired' based on the date and time.

I admit that this point is a bit semantic. But it is just to prevent people from taking data from different data sets, e.g. twitter messages from Donald Trump or Elon Musk, and daily positions of the stock exchange, and assume that there is no dependency if there is no explicit pairing (the pairing is not clear since the data has different dimensions, but you can still relate the data samples in some more complex way than pairing).

$\endgroup$
6
  • 1
    $\begingroup$ You quoted one line from my original post, so I want you to clarify a bit. I also said that "... there is no one element of $(X_1,\ldots,X_n)$ such that we can 'match' it with some element of $(Y_1, \ldots, Y_m)$". Perhaps this is a poor wording, so I will try to rephrase it below. $\endgroup$
    – Rodvi
    Oct 12, 2021 at 13:34
  • 1
    $\begingroup$ Consider two random samples $(X_1, \ldots, X_n) \overset{\text{iid}}{\sim} F_X$, $(Y_1, \ldots, Y_m) \overset{\text{iid}}{\sim} F_Y$ which satisfy the following requirement - for any possible their realizations $(x_1,\ldots,x_n), \,(y_1,\ldots,y_m)$ we will not be able to find at least one element from $(x_1,\ldots,x_n)$ which can be somehow matched (or be paired) with some element from $(y_1,\ldots,y_m)$. I think we call such realizations $(x_1,\ldots,x_n)$ and $(y_1, \ldots,y_m)$ unpaired/independent samples (or unpaired/independent data). $\endgroup$
    – Rodvi
    Oct 12, 2021 at 13:34
  • 1
    $\begingroup$ So, using this extra information (i.e. not only information that $(X_1, \ldots, X_n) \overset{\text{iid}}{\sim} F_X$, $(Y_1, \ldots, Y_m) \overset{\text{iid}}{\sim} F_Y$), can we say that the random variables $X_i$ and $Y_i$ are statistically dependent or independent? $\endgroup$
    – Rodvi
    Oct 12, 2021 at 13:35
  • 2
    $\begingroup$ @Rodvi I have removed the passage. Regarding your last question: The independence between samples occurs if the outcomes of the two variables are unrelated. So if $F_Y$ is not dependent on the $X_i$ and $F_X$ not on the $Y_i$. Gathering samples pairwise, such that variables might have some relation (even when it is only remotely), is one practical setting where variables might have a dependency. Another way is when the sample $X$ influences $F_Y$ not in terms of a pairwise relationship, but if the entire sample influences $F_Y$. For instance, the parameters in $F_Y$ could depend on $\sum X_i$. $\endgroup$ Oct 12, 2021 at 14:29
  • 2
    $\begingroup$ @Rodvi I have incorporated it into my question and changed my answer from 'yes' to 'no'. $\endgroup$ Oct 12, 2021 at 15:06
5
$\begingroup$

Independence has a formal definition, i.e. $f_{X,Y}(x,y)=f_X(x)f_Y(y)$.

On the other hand, I do not think that the term 'unpaired samples' has a formal statistical definition, and, at the risk of sounding snobbish, it is not a term that I ever hear formally trained (bio)statisticians use. Non-statisticians I work with will use that term, and I encounter the phrase in medical journals; when I encounter it, I assume that the speaker who uses this term is referring to the manner in which the data were collected, i.e. that the measured variables were not matched when they were obtained. So, to me, the term only makes sense in the context of a study design.

However, if I take your question at face value, then I think the answer is 'no': if all you tell me is that $X$ and $Y$ are generated from marginal distributions $F_X$ and $F_Y$, respectively, and you don't tell me anything about the joint distribution of $X$ and $Y$, then I cannot tell you whether they are independent.

Stepping onto my soapbox The 'unpaired t-test' is another misnomer and a term that I never hear statisticians use, despite the popularity of the term in the literature. Just call it a two-sample t-test. Stepping off my soapbox

$\endgroup$
1
  • 1
    $\begingroup$ As you said, the marginal distributions don't tell you about the joint distribution. One can think on this in terms of copula theory. $\endgroup$
    – Galen
    Oct 11, 2021 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.