How does a CNN (or DNN) pick out what node is associated to what class (if # of classes > 1) to minimize its error?

Question

Say we were to have a multi-classification problem with 10 classes. Our feature extractor is ready and good to go.

We train the model and all is good, but how does the model know what the true target value for each output node is in order to minimize its error?

In other words, if we have 10 output nodes lined up on top of each other (visually), then which one of these 10 nodes would correspond to class 1? (and the same questions is posed for the rest of the classes ideally)

Is this association done by the NN automatically as it's training or is there something influencing the order in which the classes are being associated with the output nodes?

Answer 1

It really depends on how you set up your one-hot encoding.

For each set of inputs to the network $\textbf{x}_i$, you have a corresponding classification $\textbf{y}_i$ that's know in advance.

Essentially $\textbf{y}_i$ is a vector of 1's and 0's. If you had three classes (A, B and C), and you're interested in class A, you could represent this as $\textbf{y}_i = (1, 0, 0)$. Class B could be $\textbf{y}_i = (0, 1, 0)$, which then means class C has to be $\textbf{y}_i = (0, 0, 1)$. Another way to interpret this is that class A points to the first node in the output layer, class B points to the second node, and the class C points to third and final node.

However, there's nothing stopping you from saying class A is $\textbf{y}_i = (0, 1, 0)$, class B is $\textbf{y}_i = (0, 0, 1)$ and class C is $\textbf{y}_i = (1, 0, 0)$.