What's the loss function of neutral network with a vector output? We know that usually the output of neutral network is a value. If the output is a vector such as the multi-steps prediction in time series, what's the loss function of vector? Is it simply the sum of the loss functions of each component?
 A: Depends on the loss function. Some losses are defined in terms of vectors, and then give a scalar for each vector. Some will just compute a scalar loss for each element and then sum/average the elements to give a single loss value.
Triplet losses are used to train embedding vectors $f(x_i)$, so for each input (an image, a text, something else), the network returns a vector. The loss $L$ for a sample $x^a$ is computed in terms of distances to the embeddings of two other samples $x^p$ and $x^n$ .
$$\begin{aligned}
L &=  \max\left\{0,
\left\|f(x^a) - f(x^p) \right\|_2^2 - 
\left\|f(x^a)-f(x^n)\right\|_2^2 
+\alpha\right\} 
\end{aligned}
$$
See also In training, I first have a solid drop in loss, but eventually the loss slowly but consistently increases. What could cause this?
Other losses just suppress the distinction between vector and scalar values. Suppose that your network takes an $n \times m$ pixel image as an input and gives an $n \times m$ matrix output as a prediction (for instance, if you're computing a boolean mask of the input image). For some task, it might be sufficient to flatten the output and the mask (label array) and compute the softmax loss in the usual way.
