1
$\begingroup$

I have a dynamic range of 256x256 matrices. I want to have a CNN based binary classifier. The matrices are images with a very wide range of intensities (10 orders of magnitude).

I am afraid to use Mean Normalization or Min-Max Scaling, because it will introduce negative values into the feature space:

$x- \bar{x} \over \sigma$ or $\frac{x - \bar{x}}{\text{max}(x)-\text{min}(x)}$

Assume I am using z-score normalization. All the negative values will go to zero after the first activation layer. Isn't it true? So basically I am loosing all the information in the regions at which the signal is bellow the average.

$\endgroup$
11
  • $\begingroup$ Min-max scaling scales to [0, 1] (but the usual formula is that you subtract min instead of mean), so does not introduce negative values. Why negative values are a problem? $\endgroup$
    – Tim
    Oct 16, 2018 at 7:14
  • $\begingroup$ @Tim, but it's sensitive to outliers. My intuitions says that all the CNN filters will be sensitive to the sign. For example assume a standard dog/cat classification problem. Now, we will extend the inputs by adding also the $-1\cdot$ images. Will the training converge into something stable? $\endgroup$
    – 0x90
    Oct 16, 2018 at 7:19
  • $\begingroup$ The second formula does not scale to [0,1], it would have to be $[x-\mathrm{min}(x)] / [\mathrm{max}(x) - \mathrm{min}(x)]$. Is that what you actually had in mind? $\endgroup$ Oct 16, 2018 at 7:19
  • $\begingroup$ @JanKukacka, yes. because the min could be very small. Although maybe [x−min(x)]/[max(x)−min(x)] is also a good idea. What if I create 3 channel inputs with these 3 normalization? $\endgroup$
    – 0x90
    Oct 16, 2018 at 7:21
  • $\begingroup$ There is no requirement for neural networks that inputs need to be positive... You have weights and bias terms everywhere, so if you multiplied inputs with -1, then to get equivalent weights multiply them by -1, if you shift inputs by constant, adjust bias etc. The role of scaling is to have inputs with same magnitude. $\endgroup$
    – Tim
    Oct 16, 2018 at 7:40

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.