0
$\begingroup$

It seems many kernels for Gaussian processes use magnitudes of input vectors, such as the RBF. If my inputs are age, weight (in grams), height (in mm), and my output is annual salary, then it would seem that height and weight would dominate my kernel, whereas one would expect age to play a more significant role. Is there a way to get around this issue? Most Gaussian process algorithms I have seen don't have a weighting parameter in them.

$\endgroup$

1 Answer 1

0
$\begingroup$

I would scale/center the features just as in a cluster analysis, e.g. kmeans. The distance between all features is equal after scaling, so no feature would dominate in your kernel/SVM. In theory it should help and it is in general the same process, because in cluster analysis unscaled features face the same problem, because the algorithm would not bother about e.g. the x-axis, if the x-axis ranges from 0-5 and the y-axis from 1000-60000. With scaling the algorithm would no longer be distorted, by higher scales just like your weight or height.

This SO post states exactly as you say that the rbf can have some problems with higher scales of some features. Feature scaling in svm: Does it depend on the Kernel?

So I would first try to scale my features. The question is, if you already did that, and still have those problems. I hope that answers your question, if I understand it right.

$\endgroup$
5
  • $\begingroup$ What is a good method for feature scaling? I see many options on wikipedia, so is there one that performs the best? $\endgroup$
    – Rixcrix
    Mar 28, 2021 at 6:52
  • $\begingroup$ You can rely on the normal z-transformation for scaling like in here: scikit-learn.org/stable/modules/generated/…, where quote: ' z = (x - u) / s where u is the mean of the training samples or zero if with_mean=False, and s is the standard deviation of the training samples or one if with_std=False', There are some specifics, e.g. the MinMax-Scaler can not deal with outliers very good etc. if you are interested in this check this link: towardsdatascience.com/… $\endgroup$ Mar 28, 2021 at 9:53
  • $\begingroup$ However, there is no canonical answer, I saw a lot people using the standard scaler, but as always in ML you are free to choose which scaling helpsyour model better, but in the end I believe other hyperparameters have more influence, e.g. in SVM gamma(sigma) or the C parameter $\endgroup$ Mar 28, 2021 at 9:55
  • $\begingroup$ Great thank you for your help! $\endgroup$
    – Rixcrix
    Mar 28, 2021 at 18:38
  • $\begingroup$ then i would appreciate an upvote! $\endgroup$ Mar 28, 2021 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.