What is the loss function used for CNN? For example, in AlexNet, they never specified what loss function they were using.
https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
The output is a probability vector of dimension $1000 \times 1$
So either they are using Euclidean distance with one-hot encoded cateogries.
Or some multi-class logistic regression loss.
Can someone help?
 A: In most cases CNNs use a cross-entropy loss on the one-hot encoded output. For a single image the cross entropy loss looks like this:
$$
- \sum_{c=1}^M{(y_c \cdot \log{\hat y_c})}
$$
where $M$ is the number of classes (i.e. $1000$ in ImageNet) and $\hat y_c$ is the model's prediction for that class (i.e. the output of the softmax for class $c$). Due to the fact that the labels are one-hot encoded and $y$ is a $(1000 \times 1)$ vector of ones and zeroes, $y_c$ is either $1$ or $0$. Thus, out of the whole sum only one term will actually be added: the one with $y_c=1$. 
A: As Jan says in a comment, AlexNet uses cross entropy as the loss function.
It's important to note, though, that a Convolutional Neural Network describes the architecture of the network, not the goal of the network. It is the goal of a network that determines the loss function.
CNN architectures can be used for many tasks with different loss functions:


*

*multi-class classification as in AlexNet


*

*Typically cross entropy loss


*regression


*

*Typically Squared Error loss


*image segmentation


*

*Can use cross entropy loss  as well, but can also use several other kinds of loss functions


*reinforcement learning


*

*In Deep Q-Networks, the "Expected discounted accumulated future reward" can be used


*generative adversarial networks (generating images)


*

*The Jensen–Shannon divergence was used in the original implementation


