1
$\begingroup$

I am using Iris dataset and trying to scale the feature to the range [0,1]. After normalization, I want to binarize the feature. The iris database contains n = 150 examples, each of length d = 4 features. One method is to normalize using the standard deviation applying the formula :

scaled_data = data - mean of data /standard deviation of data

Implementing it gives,

iris_data = load('iris.mtx');
y = load('iris.truth');
y = y + 1;
[n,d] = size(iris_data);
for d = 1:d
scaled_data(:,d) = (iris_data(:,d)- mean(iris_data(:,d))/std(iris_data(:,d))
end

As an example, I generated random numbers from the Uniform distribution in the range [0.1,9] as

a = 0.1;
b = 8;
r = (b-a).*rand(1000,1) + a;
scaled_data = (r - mean(r))/std(r);

But, this does not scale the values to [0,1] !! The output of scaled_data is in [-1.75,1.7] But, I am unsure if this is the correct way to do so - should I scale across the rows or across the columns and how?

Problem 1: Please help in how to properly scale the feature to the [0,1] range . How to normalize data to 0-1 range?

explains the procedure 

data = (data - min(data))/(max(data)-min(data));

but what is the name for this procedure and for a feature database, should I be scaling using each example or each feature variable?

Problem 2: then to convert to 0/1.

Few samples from the iris dataset are:

iris_data = 

5.10000000000000    3.50000000000000    1.40000000000000    0.200000000000000
4.90000000000000    3                   1.40000000000000    0.200000000000000
4.70000000000000    3.20000000000000    1.30000000000000    0.200000000000000
4.60000000000000    3.10000000000000    1.50000000000000    0.200000000000000
5                   3.60000000000000    1.40000000000000    0.200000000000000
Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

First: normalizing data is usually done per feature, that is, all instances of one feature (1 column of the 4 or 5 columns in iris) over all samples (150 rows in iris). So, normalizing over each individual column is correct in your example. To be specific, for normalization by the mean (usually referred to as "shift"), one step would be: take e.g. the Sepal.Length feature and compute the mean over all 150 Sepal.Length values in your dataset. Then divide all 150 Sepal.Length values by exactly this value.

Normalizing data by the mean and standard deviation (SD) will transform data so that its new mean=0 and SD=1: so the range is intentionally not [0,1], which is correct and OK for many models. Further, a [0,1] range instead of normalization by mean and SD has the downside that it is unstable, as strongly influenced by any outliers. Those outliers will determine the shift/scale of your data - instead of the "majority" of the data - therefore might prevent your transformed data looking anything close to a normal distribution, which would usually be desirable.

For your example in Matlab/Octave$^1$: you seem to mix / (matrix-wise operation) and ./ (piecewise opertation). The first example snippet normalizes iris_data to mean=0 and SD=1 (be aware that I only used iris column 1 to 4, as column 5 is categorial)

> iris_data2 = (iris_data - mean(iris_data)) ./ std(iris_data);
>  mean(iris_data2)

ans =

  -1.4572e-15  -1.6383e-15  -1.2923e-15  -5.5437e-16

> std(iris_data2)

ans =

   1.00000   1.00000   1.00000   1.00000

This second example snippet one does the same, but using a target range of [0,1] instead:

>  iris_data3 = (iris_data - min(iris_data)) ./ (max(iris_data)-min(iris_data));
> max(iris_data3)

ans =

   1   1   1   1

> min(iris_data3)

ans =

   0   0   0   0

For the question on how to name those: I would usually refer to them as "normalization to mean=0 and SD=1" and "normalization to range [0,1]", as it is usually clear enough what is meant by it and how it is done on the data - but this might be different for your audience (of this statement).

For binarization of features: this requires a threshold to be defined: values below and above it will become 0 and 1 through binarization. With a range of [0,1], a threshold of 0.5 could be reasonable (could be adapted to better suit your needs):

> iris_data4 = iris_data3 > 0.5;
> iris_data4

iris_data4 =

   0   1   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0
   0   1   0   0
   [...]

$^1$ I tried it with Octave, so if there should be problems in Matlab please let me know.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for your answer to the first part. My other part is that after normalization to range [0,1] how to binarize the features? $\endgroup$
    – Srishti M
    Commented Jul 3, 2016 at 10:40
  • $\begingroup$ @SrishtiM true, overlooked that part - I've updated the answer. $\endgroup$
    – geekoverdose
    Commented Jul 3, 2016 at 10:48
  • $\begingroup$ Thank you very much. One last Question although it is not a part of this post but related. If you could help - I would use neural network for classification and pretty new to the field. Would the neural network have 4 inputs (one for each column / feature) and 2 outputs : 00 denoting class1, 01 denoting class2,10 denoting class3? The input to the neural network would be then row-wise? I can put up a new Question if the answer is a detailed one, asking on how to apply neural network after this post-processing step. $\endgroup$
    – Srishti M
    Commented Jul 3, 2016 at 17:31
  • $\begingroup$ @SrishtiM This would in fact be another question (open one if you get stuck somewhere). In short: yes, row-wise input, but you would use one-hot encoded class outputs (1 of $N$ output variables for $N$ classes being 1, the others 0). Note that many ML tools encode output like this automatically, given one categorial class label - so you might not need to do this yourself. $\endgroup$
    – geekoverdose
    Commented Jul 3, 2016 at 17:43
  • $\begingroup$ Sorry to sound noisy. I have one last confusion in the binary step. So, are we binarizing taking the mean of each sample which would be a scalar value i.e., for each example (row-wise) I calculate the mean of the sample and then use that as a threshold Or will mean be calculated for each feature component? Also, I have posted a new Question stats.stackexchange.com/questions/222104/… asking about the training and the structure of the neural network. $\endgroup$
    – Srishti M
    Commented Jul 4, 2016 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.