# Backpropagation with binary cross entropy loss formula

I will classify using a neural network algorithm. I use 2 output, Y1=1 (positive) and Y2=0 (negative). The architecture is as follows:

loss that I use is binary cross entropy with the following formula:

From the loss, the error gradient with respect to weight v is:

based on the architecture that I demonstrated, is it correct to calculate the error gradient with respect to v11 like this?

• where is the architecture? Commented Jan 9, 2023 at 7:51
• @gunes oow sorry, I just added Commented Jan 9, 2023 at 7:58
• what are the activations? Commented Jan 9, 2023 at 8:52
• It's sigmoid biner Commented Jan 9, 2023 at 10:04
• For a binary classification problem (two mutually exclusive classes), either you only use 1 output neuron, or you have to use an activation on the output $y_1, y_2$. But you have 2 units, and I don't see an activation function for $y_i$. The problem is that, as it's written, you have $\log y_i$, but in the absence of an activation, it's possible (even likely) to have $Wz + b \le 0$, which means that $\log y_i$ is not a real number. If you mean $y_i=\sigma(W_kz + b_k)$, then it is not necessarily true that $y_1 + y_2 = 1$, so the outputs are not probabilities of mutually exclusive events.
– Sycorax
Commented Jan 9, 2023 at 14:21

According to the architecture, the forward and backward propagation equations are: \begin{align}Z_j&=\sigma\left(\sum_{i=1}^{n_{input}} X_iV_{ij}+b_j\right)\rightarrow \frac{\partial Z_j}{\partial V_{ij}}=X_iZ_j(1-Z_j) \\Y_k&=\sigma\left(\sum_{j=1}^{n_{hidden}} Z_jW_{jk}+b_k\right)\rightarrow \frac{\partial Y_k}{\partial Z_j}=Y_k(1-Y_k)W_{jk} \\\frac{\partial Y_k}{\partial V_{ij}}&=\sum_{j'=1}^{n_{hidden}}\frac{\partial Y_k}{\partial Z_{j'}}\frac{\partial Z_{j'}}{\partial V_{ij}}=\frac{\partial Y_k}{\partial Z_{j}}\frac{\partial Z_j}{\partial V_{ij}}=Y_k(1-Y_k)W_{jk}X_iZ_j(1-Z_j) \\E&=-\sum_{k=1}^{n_{out}}Q_k\log Y_k+(1-Q_{k})\log(1-Y_k)\rightarrow \frac{\partial E}{\partial Y_k}=\frac{Y_k-Q_k}{Y_k(1-Y_k)}\end{align}
Using the chain rule, we have \begin{align}\frac{\partial E}{\partial V_{ij}}&=\sum_{k=1}^{n_{out}}\frac{\partial E}{\partial Y_k}\frac{\partial Y_k}{\partial V_{ij}}=\sum_{k=1}^{n_{out}}(Y_k-Q_k)W_{jk}X_iZ_j(1-Z_j)\end{align}
And, substituting $$i=j=1$$ here yields the following: \begin{align}\frac{\partial E}{\partial V_{11}}&=\sum_{k=1}^2(Y_k-Q_k)W_{1k}X_1Z_1(1-Z_1)\\&=(Y_1-Q_1)W_{11}X_1Z_1(1-Z_1)+(Y_2-Q_2)W_{12}X_1Z_1(1-Z_1)\end{align}
• If we compute $y_i = \sigma(z w_i + b_i)$ as the ordinary sigmoid function, then it is not necessarily true that $y_1 + y_2 = 1$. Does that complicate this result, or the loss computation, at all?