4
$\begingroup$

I have a basic understanding of kernel methods and the kernel-trick and the advantages of it, why it is preferred over conventional machine learning algorithms etc. However, I have some trouble using them. The problems I face are as follows,
1. can I use a kernel metric for (dissimilarity) calculation?
2. what steps need to be taken for using a kernel method (say, using the gaussian kernel) on a set of categorical (along with numerical) data. consider the following sample data
Age State Day

12 NJ Tue
24 NM Wed
89 CA Thu
. . .
. . .

The question is do I need to explicitly encode the categorical values in order to use it on the gaussian kernel for similarity calculation?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

You have several options, the best one being using a kernel function that is tailored to your specific problem. This is also the least intuitive option, so I won't elaborate on that.

Usually, categorical data are encoded using so-called one-hot encoding. This means we introduce one binary feature for every category. For instance, assume we have 3 categories $A$, $B$ and $C$: $$ A \rightarrow [1, 0 ,0],\quad B\rightarrow[0,1,0],\quad C\rightarrow[0,0,1] $$ The crucial thing is that we must ensure that all pairwise distances are equal, otherwise the distance will be biased towards some pairs (unless, ofcourse, this is what you want). One-hot encoding is the simplest way to ensure that all pairwise distances between categories are equal.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Assuming that the pairwise similarity is equal (if we want it so ) can we use this as the input for a kernel method, once we use bitmap-indexing (or one-hot encoding as it is named)? $\endgroup$
    – damith219
    Commented Nov 16, 2014 at 12:28
  • $\begingroup$ @damith219 yes, you can use the binary representation as input. $\endgroup$
    – Marc Claesen
    Commented Nov 16, 2014 at 12:33
0
$\begingroup$

  1. No, by definition, kernel is some kind of similarity not dissimilarity.

  2. Basically, kernel is a way to express your domain knowledge of the data. For discrete values, you'd better use a kernel that is suitable for the data. Always using Gaussian kernel no matter what data you have is a bad practice.

BTW, kernel methods themselves do not make much sense.

Improve this answer
$\endgroup$
4
  • 8
    $\begingroup$ Statements like 'kernel methods do not make much sense' are not very useful without at least some justification. Given the vast amount of research being done on this particular topic, it's fair to say many experts disagree. $\endgroup$
    – Marc Claesen
    Commented Nov 15, 2014 at 23:14
  • $\begingroup$ You are right. It is just a personal opinion. $\endgroup$
    – Mo Chen
    Commented Nov 15, 2014 at 23:17
  • $\begingroup$ Any measure of similarity can be turned into a measure of dissimilarity. E.g. by multiplying with $-1$. $\endgroup$
    – bayerj
    Commented Nov 16, 2014 at 10:25
  • $\begingroup$ I agree with Marc. The kernel trick, for example uses the kernel itself as an effective classification method without employing any other machine learning technique, if I am correct in understanding it. $\endgroup$
    – damith219
    Commented Nov 16, 2014 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.