In multi-class logistic regression, lets say we use softmax and cross entropy. Does SGD one training example update all the weights or only a portion of the weights which are associated to the label ?
For example, the label is one-hot [0,0,1]
Does the whole matrix W_{feature_dim \times num_class} updated or only W^{3}_{feature_dim \times 1} updated ?
The whole matrix W will be updated. In particular, all weights associated with target class will increase (for all features), and all other weights will decrease. (So in your example, all elements in third column of the matrix will increase, and all other elements will decrease). It follows from the derivative of softmax function.
edit: Weights will increase / decrease as I wrote only if features are all positive. For features with negative values, the weight update will go in the opposite direction.
