# How to derive categorical cross entropy update rules for multiclass logistic regression

I am aware that there is a similar post: Vectorization of Cross Entropy Loss

But looking it over for a couple hours, I still do not understand what they are trying to say.

I'm trying to find the update rules for using the cross entropy function in multiclass logistic regression, here for 10 classes.

I'm confused what this is even implying - is this the cost for the entire dataset or just the cost for a single example? I'm not sure if y and yhat are scalars and the summation goes through a single example or that y and yhat are vectors and that the i indexing means it's going through the entire set.

Assuming that it's going through the whole training set, how do we derive it w.r.t all the thetas? how to vectorize it, and to come up with a general update rule for all weights?

I would be so grateful for an answer - I've tried with pen and paper for a long time and haven't gotten it yet.

Thanks, A

This is not over the entire the training set. It's for a single example, $$i$$ represents the class index. So, if you have $$n$$ classes, $$\mathbf y$$ and $$\mathbf{ \hat y}$$ are $$n\times 1$$ real vectors. $$\mathbf y$$ is the ground truth, so it'll be a binary vector with the index corresponding to the correct class for the sample being $$1$$ and others being $$0$$.
To find the gradients, you can assume one sample only, and then sum the gradients for each training sample. So, for simplicity, do the differentiation using this formulation, i.e. $$\frac{\partial L}{\partial \theta}$$, where $$y=\sigma( \theta^Tx+b)$$.