2
$\begingroup$

I have been looking at some tutorials and articles and couldn't get a scenario where two variables are in different scales and used in modeling.

So, firstly lets assume I have one metric of numeric type, other in percentages, and other in decimals.

  1. If I want to use those variables in a regression model for prediction then do I need to do some standardization before fitting a
    model to the variables? If so how do we it in R or Python?
  2. Moreover, if I want to use these features in k-means clustering, do I need to follow the same steps as mentioned above?
Improve this question
$\endgroup$
  • $\begingroup$ Welcome to SO. Your question(s) seem to be (especially the second one) related to statistics not programming. Next time try to provide a piece of code here how to do it, and try to do not ask many questions in one SO question. $\endgroup$ – SabDeM Aug 2 '15 at 5:01
7
$\begingroup$

For a linear regression you do not have to standardise. You just have to take care with the interpretation of the estimated coefficients. Assume e.g. that you have two independent variables, $x_1$ in meter and $x_2$ in kilometer and you estimate the regression model $y = \beta_1 x_1 + \beta_2 x_2 + \beta_3 + \epsilon$, then there is no problem with the estimates $\hat{\beta}_i,i=1,2,3$.

However you should not compare the coefficients in order to draw conclusions on the impact of the variables, so e.g. if $\hat{\beta}_1 > \hat{\beta}_2$ you can not say that $x_1$ has more impact on y than has $x_2$ because $x_1$ is in meter, while $x_2$ in km.

If you want to standardise $x_1$ in R then you compute 'x1.standardised<-(x1-mean(x1))/sd(x1)'.

Cluster analysis is different because it is based on a so-called (di-)simmilarity matrix: it clusters together points that are close to each other (=that are similar to each other). The elements of this matrix represent the (inverse) distances between any two points in your sample.

The distance measure used in cluster analysis does not have to be the Eucilidean distance but let's take it as an example. Then if I measure the distance between two datapoints where the first coordinate is in meter and the second one in km, then the size of the difference between the coordinates of the two points will probably be larger for the first coordinate because it is expressed in meter.

The Euclidian distance is $\sqrt{\text{difference of the first coordinates}^2 + \text{difference of the second coordinates}^2}$ So the difference for first variable (in meter) will weight more than for the second one when we compute the Eucilidian distance, which means that the cluster analysis will give more importance to the first variable for clustering.

Improve this answer
$\endgroup$
7
$\begingroup$

for the standardizing part of your question, if you want to standardize a vector x in R, use the function 

     scale(x)

it is equivalent to 

    (x-mean(x))/sd(x)
Improve this answer
$\endgroup$
  • $\begingroup$ @fcoppens thanks for the explanation. I have a follow up question though, why do we need to standardize in k-means clustering? if you take the Euclidian distance method also based on what I understand the center will have two params x,y and another observations lets say x1,y1 and x2,y2 so x is in meters and y is in kms. however it is same for other observations too! so if we apply the euclidian distance it shouldn't be a problem right? pls. let me know $\endgroup$ – yome Aug 3 '15 at 17:10
  • 1
    $\begingroup$ @yome; the point is not that it makes a difference for the points $(x_1,y_1)$ and $(x_2,y_2)$, the point is that one of the two dimensions is will not be taken into account. Let the x's be in meter and the y's in km. if $(x,y)$ is the center of a cluster and we want to ''classify'' $(x_1,y_1)$ where $|x_1-x|=1000m$ and $|y_1-y|=1km$ then the Euclidean distance $d((x,y),(x_1,y_1))=\sqrt{1000ˆ2+1ˆ2}$ so only the differences in the first dimensions are taken into account, do I make myself clear like this ? $\endgroup$ – user83346 Aug 3 '15 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.