2
$\begingroup$

I see that in Bayesian neural networks likelihood function is defined in two ways:

$p(W|D) = Z^{-1} p(D|W)p(W)$

or

$p(W|y,x)=Z^{-1}p(y|x,W)p(W)$

Are there a slight difference in interpreting $p(D|W)$ and $p(y|x,W)$? is the first one read as the probability of our data given the parameters (weights of the NN) and the second one, the probability of the output label given the input and the parameters? to me, the first notation is more intuitive. I wonder if there is no difference among them, why author choose to go with the second one?

$\endgroup$

1 Answer 1

1
$\begingroup$

There is no "correct" notation, the point of using mathematical notation is communicating your thoughts. Notice how many details do the notation above ignores, it tells you nothing about what are those variables, where do they come from, etc. Usually the more complicated problem you are trying to communicate, the more abstract the notation gets and the more details are omitted. The same applies to the notation above. You would see both forms used commonly, sometimes meaning the same thing.

As you can learn from the What are the differences between stochastic and fixed regressors in linear regression model? thread, the features of the model ($x$'s) can be either treated as deterministic or as random variables. If they are deterministic, you technically cannot condition on them, so $p(y|x,W)$ does not make sense. If you treat $x$ as deterministic, it doesn't go into $p(\cdot|\cdot)$ since it is only about random variables. To be more precise, sometimes people use special notation to denote that the value is non-random. So there are scenarios where such details may be important.

Sometimes people use notational tricks or are just sluggish. In each case, you should read the notation in the context of the rest of the paper or book, to make sense of it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.