2
$\begingroup$

Are the concepts of normalizing and scaling of data in conflict with each other?

I am adding weights to my features, I have tried normalizing the weights and it didn't make any difference in the outcome. I have also scaled my input data. and got positive results.

however, I have heard from other sources, fellow students- not so reliable, that I should scale the weights.

So I am rather stumped, I don't know if it is something really simple and I am having a mental block or it it is something more complex that I am not understanding.

|cite|improve this question
$\endgroup$
  • $\begingroup$ scale the weights or what? What algorithm are you using? $\endgroup$ – Charlie Parker Nov 1 '15 at 22:57
4
$\begingroup$

You should not touch the weights. The way to proceed is to center and scale the training data and apply that same transformation to the test data.

They complement each other. You may have a look at this video where the topic is nicely explained. Scaling is relevant from a practical point of view, because large scale methods are sensitive to unnormalized data.

Notice that SVMs are based on scalar products. If the scales of different features vary widely, (for example one has mean 1000 and variance 1000000) and another one is centered around 0 and variance 10, then the first is likely to drive the scalar products if not scaled. This idea also applies to gradient calculation (which is relevant for solvers).

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Thank you. I usually get stuck in some aspect of my work and lose all hope! I just found this place yesterday and everyone who has answered my questions has been so helpful. And thanks for the video link too! $\endgroup$ – Kate Jul 19 '14 at 13:23
  • 1
    $\begingroup$ Everyone feels like that every now and then. You're not alone!. I learn a lot here. And the best is to get a couple of good references and keep them by your side. And, of course, study them! :) $\endgroup$ – jpmuc Jul 19 '14 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.