0
$\begingroup$

From Alex Graves Ch. 2:

$(x,z)$ are input, target pairs in the training set $S$, and $S$ is drawn from the input-target distribution $D$. I am confused by the following line:

$p(S|w)=\prod_{(x,z)\in S}^{ }p(z|x,w)$

Is $p(S|w)$ the density $(x,z)$ conditioned on parameter $w$? The RHS of the expression suggests that $p(S|w)$ is the likelihood function $L(w)=p(z|x;w)$, or the probability of observing $z$ conditioned on input $x$ and parameters $w$. However, I fail to see how $p(S|w)=p(z|x;w)$. $x$ is being treated as given in one, but not the other.

$\endgroup$
0
$\begingroup$

In this context, I'm guessing $w$ is some shape parameter describing the relationship between $z$ and $x$ (for example, maybe $z \sim N(w,x)$). The goal of the likelihood maximization is to produce an estimate $\hat w$ of $w$. For this, only the relationship of $z$ to $x$ is what matters. The distribution of $x$ itself is a priori information -- the likelihood maximization doesn't have to describe the true distribution $x \sim X$ or the likelihood of getting the observed values $x_i$. The observed values of $x$ are given; they are what they are. What the likelihood does care about is how the observations $z$ depend on the given $x$, and so it depends on $p(z | x,w)$, which is the conditional probability of z conditioned on $x$ and $w$. In the normal distribution example I gave above, this would just be a gaussian pdf $$ p(z|x,w) = \frac{1}{x\sqrt{2\pi}} e^{-\frac{(z-w)^2}{2x^2}} $$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.