4
$\begingroup$

For iid random variables $Y_1,\dots,Y_n$ from a random sample that are mutually independent with pdf $f(y|\theta)$, my book defines the likelihood as $L(y_1,\dots,y_n|\theta)= f(y_1,\dots,y_n|\theta) = f(y_1|\theta)\dots f(y_n|\theta)$. For a random sample of the form $(X_1, Y_1) \dots (X_n,Y_n)$ with conditional densities $f(y|x,\theta)$, I've seen other sources define the conditional likelihoods as $\Pi_{i=1}^{n} f(y_i|x_i,\theta)$.

Based on how the likelihood is defined in my book, I interpret the conditional likelihood as a joint probability distribution $f(y_1,\dots,y_n|\theta,x_1,\dots,x_n)$ (written as $L(\theta; y|x)$ in some sources). If this is correct, is requiring that $Y_1,\dots,Y_n$ be mutually independent sufficient to define the likelihood as $\Pi_{i=1}^{n} f(y_i|x_i, \theta)$? I thought for the product be valid, $f(y_i|x_i, \theta)$ must equal $f(y_i|x_1, \dots, x_n, \theta)$ which implies that $Y_i$ and $X_1,\dots,X_{i-1},X_{i+1},\dots,X_n$ are conditionally independent given $X_i$? i.e. that $f(y_i,x_1,\dots,x_{i-1},x_{i+1},x_n|x_i)=f(y_i|x_i)f(x_1,\dots,x_{i-1},x_{i+1},x_n|x_i)$.

I haven't been able to find a source that explicitly state that $Y_i$ and $X_1,\dots,X_{i-1},X_{i+1},\dots,X_n$ be conditionally independent given $X_i$; is this implied or am I misunderstanding something?

$\endgroup$

1 Answer 1

2
$\begingroup$

Normally, iid assumption is for the pairs, not just outcome or feature RVs, i.e. $(x_1,y_1)...(x_n, y_n)$ are iid pairs in a dataset. This may not be explicitly stated in some sources. This assumption directly means $Y_i$ and $X_{-i}$, i.e. variables other than $X_i$, are conditionally independent given $X_i$.

Here is another topic which might be quite useful.

$\endgroup$
2
  • $\begingroup$ Based on the link, iid for pairs assumes $p(x_1,y_1,\dots,x_n,y_n)=f(x_1,y_1)\dots f(x_n,y_n)$ ($p$ and $f$ being joint densities)? If so, does this assumption mean that $Y_i$ and $X_{-i}$ are mutually independent and also that all the $X$'s are mutually independent from each other, which is stronger than just assuming $Y_i$ and $X_{-i}$ are conditionally independent given $X_i$ per this answer? $\endgroup$
    – Yandle
    Commented Apr 5, 2020 at 16:51
  • $\begingroup$ I have seen that in some cases conditional independence type assumptions are made instead of pairs being independently sampled. I was also wondering what is the exact description of the weaker conditional independence assumption, for example, in the probabilistic motivation of least squares (see this question). $\endgroup$
    – Dylan Dijk
    Commented Jun 9, 2023 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.