That's because the author is considering in that part that the activation function of a neuron $j$ only depends on the value of $z_j^L$. This happens for example with the sigmoid or ReLU activation functions.

However, when using activations functions like Softmax (described in Chapter 3.1.4), we have that: $$a_j^L = \frac{e^{z_j^L}}{\sum_k e^{z_k^L}}$$ Here we can see that $\partial a_j^L/\partial z_k^L \neq0$ (or equivalently: $\partial a_k^L/\partial z_j^L \neq0$) for $k\neq j$, so the correct computation of $\delta_j^L$ is the equation given by the author: $$ \delta_j^L=\sum_k\frac{\partial C}{\partial a_k^L}\frac{\partial a_k^L}{\partial z_j^L} $$

So, to sum up, the above equation is the general expresion of $\delta_j^L$, however if we use certain activation functions, it can be simplified to the equation $(38)$ of the question.

That's because the author is considering in that part that the activation function of a neuron $j$ only depends on the value of $z_j^L$. This happens for example with the sigmoid or ReLU activation functions.

However, when using activation functions like Softmax (described in Chapter 3.1.4), we have that: $$a_j^L = \frac{e^{z_j^L}}{\sum_k e^{z_k^L}}$$ Here we can see that $\partial a_j^L/\partial z_k^L \neq0$ (or equivalently: $\partial a_k^L/\partial z_j^L \neq0$) for $k\neq j$, so the correct computation of $\delta_j^L$ is the equation given by the author: $$ \delta_j^L=\sum_k\frac{\partial C}{\partial a_k^L}\frac{\partial a_k^L}{\partial z_j^L} $$

So, to sum up, the above equation is the general expresion of $\delta_j^L$, however if we use certain activation functions, it can be simplified to the equation $(38)$ of the question.