I have a joint model where I have the output variable $Y$ and independent variable $X$ where I treat each of the output variable as being independent of each other. Additionally, there are model parameters $\beta$ and $W$. Now, I have $i$ independent observations and I write the joint log model as:

$$ \log p(Y, \beta, W|X) = \sum_{i}^{N}\log P(y_i|x_i, w_i, \beta) + \log P(\beta) + \sum_{i}^{N}\log P(w_i) $$

What I am wondering is that considering that each of the $y_i$s are independent of each other and the same for $w_i$, can I write:

$$ \log p(y_i, \beta, w_i|x_i) = \log P(y_i|x_i, w_i, \beta) + \log P(\beta) + \log P(w_i) $$

I am having this confusion as there is no sum over the prior distribution $P(\beta)$ of course, so could not convince myself this should be the case. I was wondering given this joint model with the independence assumptions that I made, whether I can write the joint model for each of the $i$ variables in any particular way?

After thinking about it, I think I can write the joint model for each $i$th term as:

$$ \log p(y_i, \beta, w_i|x_i) = \log P(y_i|x_i, w_i, \beta) + \frac{\log P(\beta)}{N} + \log P(w_i) $$

The reason I say this is because I can write the joint model as:

$$ p(Y, \beta, W|X) = P(\beta) \prod_{i}^{N} P(y_i|x_i, w_i, \beta) \prod_{i}^{N} P(w_i) = \prod_{i}^{N}P(y_i|x_i, w_i, \beta)P(w_i)\frac{P(\beta)}{N} $$

And now since $y_i$s and $w_i$ are independent, I think I can write everything as described above.

Can someone verify if this is correct or if I am on the wrong track?


Assuming what I understand from your independence hypothesis, the two first equations are good: $$ p(y_i,\beta,w_i|x_i)=p(y_i|\beta,w_i,x_i) \cdot p(\beta,w_i | x_i) = p(y_i|\beta,w_i,x_i) \cdot p(\beta) \cdot p(w_i) $$ and $$ p(Y,\beta,W|X)=\prod (p(y_i | \beta,W,X)) \cdot p(\beta,W,X) = \prod p(y_i | \beta,w_i,x_i) \cdot p(\beta) \cdot p(W)=\prod p(y_i | \beta,w_i,x_i) \cdot p(\beta) \cdot p(w_i) $$

Your last equation is good except the thrid equality as the $\frac{1}{N}$ factor should not be here (I guess that you messed with sum but $\beta \prod( \alpha_i)$ equals $\prod (\beta \alpha_i)$).

Nevertheless I understand your confusion : mixing log proba and independence is sometimes bluring. As a conclusion, decompose your probabilities before thinking of their logs (which can indeed blur our minds) : things would maybe appear clearer.

| cite | improve this answer | |
  • $\begingroup$ yes, i think I indeed made that mistake. I see it now. Thanks for your reply! $\endgroup$ – Luca Oct 29 '15 at 8:49
  • 1
    $\begingroup$ @luca I did one year ago ! and appreciate that others make it also. So thanks too. $\endgroup$ – peuhp Oct 29 '15 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.