0
$\begingroup$

I read that using convolutional neural networks, or any neural networks (?), that the input/features should be normalized. The normalization is typically done for each feature $$x_i \in X \ \ \forall i$$ with respect for a set of samples $N$.

I have a couple of questions pertaining to this:
(1) Are features ever normalized with respect to other features, or is the normalization always independent of the other features? And does this depend on the independency/dependency of the features with respect to one another?
(2) Does the normalization constant depend on the distribution (e.g., linearity, nonlinearity) of the samples for a given feature?

$\endgroup$
0
$\begingroup$
  1. See ZCA whitening, which causes the covariance matrix of the features to be identity. Although I don't see it commonly applied anymore, probably because it's not necessary.

  2. You usually want the distribution to be not too skewed. For example, it is common when dealing with depth images (where each pixel contains not the RGB value, but the distance to the pixel in 3D space) to perform some transformation (such as log).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.