I read that using convolutional neural networks, or any neural networks (?), that the input/features should be normalized. The normalization is typically done for each feature $$x_i \in X \ \ \forall i$$ with respect for a set of samples $N$.
I have a couple of questions pertaining to this:
(1) Are features ever normalized with respect to other features, or is the normalization always independent of the other features? And does this depend on the independency/dependency of the features with respect to one another?
(2) Does the normalization constant depend on the distribution (e.g., linearity, nonlinearity) of the samples for a given feature?