I want to train a neural network on a classification task, and I understood that normalizing the data helps the network to converge faster. Let's assume I normalize my data via
norm_data = (data - mean)/standard_deviation
My question is: Do I compute the mean and standard deviation of the whole input data set, or do I do this separately for every row? I found examples for both methods, and now I wonder which one is better.
Is there a better one, is it case-dependent, or is this not important at all? In my specific data, all input values can theoretically appear in approximately the same range.