I am aware that there is a similar post: Vectorization of Cross Entropy Loss
But looking it over for a couple hours, I still do not understand what they are trying to say.
I'm trying to find the update rules for using the cross entropy function in multiclass logistic regression, here for 10 classes.
I'm confused what this is even implying - is this the cost for the entire dataset or just the cost for a single example? I'm not sure if y and yhat are scalars and the summation goes through a single example or that y and yhat are vectors and that the i indexing means it's going through the entire set.
Assuming that it's going through the whole training set, how do we derive it w.r.t all the thetas? how to vectorize it, and to come up with a general update rule for all weights?
I would be so grateful for an answer - I've tried with pen and paper for a long time and haven't gotten it yet.