# change hinge loss error function with cross-entropy

I'm trying to implement cross-entropy as an error function in RBF neural networks instead of hinge loss error function. I need to find cross-entropy error for each output neuron, like hinge loss error function as you can see in the formula below

but when it comes to cross-entropy, it seems meaningless. because the formulation of cross-entropy returns 1 scaler. I look at some of the recent papers using cross-entropy and soft-max but can't find any relative information to my problem.

Is there any way to finding cross-entropy value for each neuron of output?

• The given hinge loss formula also returns a single scalar. What exactly is the issue here? – jkm Jun 18 '18 at 9:27
• @jkm assume n as the number of Classes so output layer has n output neuron. in the formula, I previously mentioned the j index, points output neurons and as you can see the formula calculate the error for each neuron. the output of equation is a vector ${e_1, e_2 ,...e_n}$. the problem is I can't find any same formula for each output neuron when I use cross-entropy. – mkafiyan Jun 18 '18 at 9:38
• ...just substitute the two conditional formulas in the bracket with a single standard cross-entropy formula over each j? – jkm Jun 18 '18 at 9:57
• @jkm if I do so, it becomes meaningless because in the first condition of formula the multiplication must be more than one. but in cross-entropy, we use soft-max which is the normal distribution. assume that we change the first condition, the formula of cross-entropy is $-\sum_{j=1}^{n}y_j \log{\widehat{y}_j^~}$ . in this formula just one term of sum have to value cause $y_j$ is a vector with just an element with the value of one. so it seems meaningless to me if separate the error of each neuron. maybe I don't get correctly waht you exactly mentioned. – mkafiyan Jun 18 '18 at 10:14
• @jkm do you mean using something like this : $\begin{cases}0 & y_j*\widehat{y}_j > 1\\z_j-y_j & other\end{cases}$ assume that z is softmax function ? – mkafiyan Jun 18 '18 at 10:22