1
$\begingroup$

I am trying to pre-process data following a statement in a paper.

They said

for the normalization, each dataset is normalized on a per channel basis with a sample range from -1 to 1 and a mean value of zero.

$X_{rescaled} = a+(b-a)\frac{X - min(X)}{max(X)-min(X)}$, following this, it can rescale data $X$ into an arbitrary interval $[a \ b]$, but it can't promise the mean is zero.

Probably I didn't get the statement here, anyone has any idea about this?

Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ You would have to perform the normalization in two parts, separate for the below average and above average values, but this will change the relationship of the data. $\endgroup$
    – user2974951
    Commented Nov 8, 2022 at 11:14
  • 1
    $\begingroup$ Maybe e-mail the authors? $\endgroup$
    – Tim
    Commented Nov 8, 2022 at 11:19
  • $\begingroup$ I e-mailed them a month ago with a few other questions, got no response. They did reply me once, maybe get tired of my endless questions:) $\endgroup$
    – Margie Shi
    Commented Nov 8, 2022 at 11:25
  • $\begingroup$ Does the quote mean that the new values are mapped to $[-1,1]$ or is this interval only the range of possible values? $\endgroup$
    – user2974951
    Commented Nov 8, 2022 at 11:27
  • $\begingroup$ @user2974951 thx! could you please explain a bit more? Do you mean first centring the data with zero mean and rescaling the two parts to [-1 0] and [0 1] respectively? $\endgroup$
    – Margie Shi
    Commented Nov 8, 2022 at 11:28

2 Answers 2

Reset to default
1
$\begingroup$

The following code in R was my attempt of to solve the problem of finding an algorithm to convert a vector of values to a new vector of values with mean of 0 and range of [-1, 1].

After one pass, it doesn't quite work. (The result will have the specified range, but the mean will be off-zero, though usually not by too much).

However, it appears that if it's applied iteratively (feeding the results back through the algorithm), it will reach the desired result within a certain tolerance.

I doubt this transformation is practically useful. And one could probably come up with a simpler process to arrive at a similar result.

Briefly, it centers A on mean of 0, and then divides this new A_center into those values above and below 0 (A1 and A2), adds a value of precisely 0 to each, and then applies a linear transformation to each of these to fit a range of [-1, 0] and [0, 1].

### RUN THIS ONCE, WITH A AS THE INPUT VECTOR ###

A = c(1,2,2,3,3,3,4,4,4,4,5,5,5,5,5,8,8,8,8,8,8,8,8,8,8,8)

A_trans = A

### RUN THIS ITERATIVELY UNTIL THE RESULTS ARE WITHIN TOLERANCE ###

A = A_trans

hist(A)

A_center = A - mean(A)

A1 = c(0, A_center [A_center < 0])

A2 = c(0, A_center [A_center > 0])

A1_scaled = ((A1 - min(A1) * (0 - -1))/(max(A1) - min(A1)) + -1)

A2_scaled = ((A2 - min(A2) * (1 - 0))/(max(A2) - min(A2)) + 0)

A_trans = c(A1_scaled[2:length(A1_scaled)], A2_scaled[2:length(A2_scaled)])

hist(A_trans)

Sum=data.frame(
  N    = c(length(A),length(A_center),length(A_trans)),
  Mean = c(round(mean(A), 2), round(mean(A_center), 2), round(mean(A_trans), 2)),
  Min  = c(round(min(A), 2), round(min(A_center), 2), round(min(A_trans), 2)),
  Max  = c(round(max(A), 2), round(max(A_center), 2), round(max(A_trans), 2)),
  CountLess0 = c(sum(A < 0),sum(A_center < 0),sum(A_trans < 0)),
  CountGreater0 = c(sum(A > 0),sum(A_center > 0),sum(A_trans > 0)),
  row.names=c("A", "A_center", "A_trans")
)

Sum
Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Although clever, your algorithm does not seem to be applicable in a production environment where train data artifacts need to have been stored earlier to restore them and re-use them on new, previously unseen data... But you do mention that I may not be practically useful. Anyway it is a clever way to achieve said results. $\endgroup$
    – bmasri
    Commented Nov 14, 2022 at 13:48
1
$\begingroup$

I believe the intended application of this preprocessing technique is to first standardize the data with mean zero then change its range to $[-1,1]$ for a neural network input. To quote this referece :

You will get better initializations if the data are centered near zero and if most of the data are distributed over an interval of roughly [-1,1]

However, in all cases, applying first a standardization technique to get zero mean and then apply a new data normalization to map your features to a certain range is meaningless... it is equivalent to normalizing only $X$ since the standardization step does not change the min/max values (check this question).

What I would suggest is only standardization to zero mean and unit variance like so

$$ X_{i}{scaled} = \frac{X_{i} - \mu_{X_i}}{\sigma_{X_i}} $$

where $\mu_{X_i}$ and $\sigma_{X_i}$ are computed for every feature $i$. Then if you really want to have a range of $[-1,1]$ and your data is Gaussian, you can simply filter them accordingly which according to a Gaussian distribution should give you about $68\%$ of your data. In the reference that I quote, they say 'most of the data' which in this case means $68\%$ if it is Gaussian. But this seems to me like an overkill since you loose too much data.

On the other hand, if your data is not Gaussian then clipping values will result in an even greater loss of data (and information). So one should choose a better mapping (e.g. logarithmic, box-cox, and so on...) to make data Gaussian-like first, then standardize to zero mean and unit variance, then clip to $[-1,1]$ if necessary. An example using a simple logarithmic transformation for non Gaussian features:

$$ X_i = log(X_i + \epsilon) $$ then $$ X_i = \frac{X_{i} - \mu_{X_i}}{\sigma_{X_i}} $$

See scikit-learn's implementation of standard scaler here and their implementation of non-linear data transformers here.

Update: I found this scikit-learn page that discusses tips to tackle deep learning data preprocessing very useful. They do indeed mention that it is recommended to either scale data to $[-1,1]$ or standardize it to have zero mean and unit variance. Applying both is meaningless as shown in this answer that @Tim so kindly provided.

Improve this answer
$\endgroup$
9
  • $\begingroup$ This doesn't work, the mean is not 0. $\endgroup$
    – user2974951
    Commented Nov 8, 2022 at 13:06
  • $\begingroup$ In fact you're right, I realize that applying first a standardization then a normalization is the equivalent of normalization alone. So I updated my answer to a more correct way to get what you are looking for $\endgroup$
    – bmasri
    Commented Nov 8, 2022 at 13:45
  • $\begingroup$ @bmasri thanks for your informative answers! I am thinking about this because my train loss is much higher (5-10 times) than the paper reported at the beginning, which is probably related to the initializations. Unfortunately, the data is not gaussian, it is ultrasound raw data, actually. I am not sure taking logarithmic make sense here. $\endgroup$
    – Margie Shi
    Commented Nov 9, 2022 at 13:56
  • $\begingroup$ @bmasri, do you think it makes sense that first change its range to $[-1 1]$ and then subtract the mean for each feature? $\endgroup$
    – Margie Shi
    Commented Nov 9, 2022 at 14:00
  • 1
    $\begingroup$ Scaling the data and then standardizing makes no sense because those transformations cancel out each other. $\endgroup$
    – Tim
    Commented Nov 14, 2022 at 14:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.