0
$\begingroup$

I have values two vectors that are spread across ranges of [-5, -1] and [1, 5] respectively. They can be decimals. Here is a toy example:

x <- c(-5, -4.5, -4.4, -3, -1.95, -1)
y <- c(1, 2, 3, 3, 3.5, 3.7)

I concatenate these to deal with one vector: z <- c(x, y) So we can see that ydoes not actually have the full range of possible values, i.e. it's largest value is 3.7, not 5.

There is nothing between -1 and 1 apart from some zeros, so nothing like 0.7.

I want to scale all the data to a [-1, 1] range, but if I use the common method of normalisation with my own function:

common_scaler <- function(input_data, new_min, new_max) {

    ## Define parameters for scaling
    old_min <- min(input_data)
    old_max <- max(input_data)
    a <- (new_max - new_min)/(old_max - old_min)
    b <- new_max - a * old_max

    ## Scale the input_data
    output_data <- a * input_data + b
    return(output_data)
}

The output is all nicely between -1 and 1, however there is a value of +1 included.

> common_scaler(z, -1, 1)
> [1] -1.00000 -0.88506 -0.86207 -0.54023 -0.29885 -0.08046  0.37931  0.60920  0.83908  0.83908  0.95402  1.00000

We know there was no +5 in the original data, so this leads me to believe we have fundamentally altered the properties of the data set. If I take the mean before and after, the interpretation will be different. For example:

> mean(z)
[1] -0.3042
> mean(scaler2(z, -1, 1))
[1] 0.0795

This changes the whole outcome for my research, i.e. the results being positive or negative.

I thought "All I want to do is scale the data down to my chosen range, so I need a factor to divide through by". I took the maximum value of the data set (so from x and y) and divide through by that, meaning my largest value should now be -1, while on the positive side, I have 0.74 (3.7/5). Here is the function, with the rsults and the comparison of means:

my_scaler <- function(x) {x / max(sqrt(x*x))}

> my_scaler(z)
 [1] -1.00 -0.90 -0.88 -0.60 -0.39 -0.20  0.20  0.40  0.60  0.60  0.70  0.74

> mean(z)
[1] -0.3042
> mean(my_scaler(z))
[1] -0.06083

This seems more reasonable to me, however I don't know if I am breaking some fundamental rules of scaling/transforming data. Have I changed the distribution in some detrimental way?

I actually have a data set with 1000 observations of 70 variables, each with their own scales. The desired result is that I have can take perform rowMeans over all the data, without getting any kind of distortion on the results. Maybe I could take the mean first, then scale all the data with my_scaler?

$\endgroup$
1
$\begingroup$

I think your my_scaler function in the last snippet will fail for the same motive the function common_scaler. What you want (depending what you want) is a linear mapping between two scales, for example $$ T:[-5,5] \to [-1,1] $$ This being linear, it's derivative is a constant, so every other linear function (mean, variance, etc.) will be scaled with the same constant.

Normally, when you scale the some N observations with m features, you don't scale every single observation independently, but scale all observations with respect to the minimum and the maximum found accounting all features. This leads to the linear mapping said before.

Now, you did the observation that your data has a minimum of -5 and a maximum of 5 and this is valid for all your data (but maybe not for every single observation), so you should be ok by just scaling by a factor 0.2, and every other linear transformation in your data should be preserved.

$\endgroup$
  • $\begingroup$ Thanks for the explanation, +1 (if not for the explanation, then for "What you want (depending on what you want) ..." :-) $\endgroup$ – n1k31t4 Dec 9 '15 at 16:41
  • $\begingroup$ My data has a potential max/min of +5/-5... there isn't always an actual plus or minus five in the data set. I have gone with dividing by the absolute of the max of the data set, so there will always be a +/-1 in the set, with all other data scaled linearly. Is this upholding with your advice? $\endgroup$ – n1k31t4 Dec 12 '15 at 22:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.