Conditional probability vs. likelihood - neural networks

Question

In Goodfellow et al.'s Deep Learning, the authors write about recurrent neural networks on page 371:

The total loss for a given sequence of $\mathbf{x}$ values paired with a sequence of $\mathbf{y}$ values would then be just the sum of the losses over all the time steps. For example, if $L^{(t)}$ is the negative log-likelihood of $y^{(t)}$ given $\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}$, then $$\begin{align} &\phantom{=} L(\{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(\tau)}\}, \{\mathbf{y}^{(1)}, \ldots, \mathbf{y}^{(\tau)}\}) \tag{10.12}\\ &=\sum_t L^{(t)}\tag{10.13}\\ &=-\sum_t \log p_{\mbox{model}}(y^{(t)} | \{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}\}) \tag{10.14} \end{align}$$ where $p_{\mbox{model}}(y^{(t)} | \{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}\})$ is given by reading the entry for $y^{(t)}$ from the model's output vector $\hat{\mathbf{y}}^{(t)}$.

Here, $\hat{\mathbf{y}}^{(t)} = \mbox{softmax}(\mathbf{o}^{(t)})$ and $\mathbf{o}^{(t)}$ is the output of the network for time $t$.

Questions:

1. Why is the likelihood of $y^{(t)}$ given $\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}$ first written as a joint likelihood over $\{\mathbf{x}, \mathbf{y}\}$ in equation (10.12)? I.e., if $\mathbf{x}$ is not being modeled how can you compute a likelihood for it? (This may be a more fundamental issue regarding my understanding of how a likelihood function relates to the probability of the data given the model.)
2. In equation (10.14), why is there no sum over the length of $\hat{\mathbf{y}}$? Or is it just saying that you only need to consider the model's predicted probability for the true category of the observation? The latter doesn't make much sense to me because, intuitively, it feels like there should be a greater penalty if the uncertainty is distributed evenly among more categories compared to if it is distributed among just a few.
3. For neural networks, how can you use a negative log probability as a surrogate for loss if noise is not being modeled?

Answer 1

1. It's not explicitly stated that $L$ is a likelihood. $L^{(t)}$ is a likelihood, but $L$ is the loss which is just a function of the entire sequence $L(\{x^{(1)},...,x^{(\tau)}\},\{y^{(1)},...,y^{(\tau)}\})$. At each time step, the sequence of $x^{(1)},...,x^{(t)}$ is observed as shown in (10.14). It might be then that $L$ is the negative log likelihood because it's just a sum of negative log likelihoods, but then would't it be something like $$L(\{y^{(1)},...,y^{(\tau)}\}|\{x^{(1)},...,x^{(\tau)}\})=-\log p_{model}(\{y^{(1)},...,y^{(\tau)}\}|\{x^{(1)},...,x^{(\tau)}\})$$

2. The label $y^{(t)}$ is a vector (as is $x^{(t)}$). For example with a binary classification problem $y^{(t)} \in \mathbb{R}^2$. The output $\hat{y} \in \mathbb{R}^2$ is also a vector of fixed length, but I believe they call it $o^{(t)}$.

3. Could you rephrase/ make 3 more formal?