I know it's common practice, but is it right to apply the common kernels to categorical/mixed data? If not, are there alternatives? I'm expecting answers from both theoretical and practical points of view.
The intended application is to any machine learning algorithm that supports the use of kernels, like (LS)-SVM, RVM, Gaussian Processes, Kernel K-NN, etc.
For all intents, consider non-binary categorical variables can be coded into binary variables, under any scheme. Solutions using all levels at once are preferred though.
Another question (Kernel methods on Categorical Data) nearly addressed the issue, but the accepted answer only mentions
[...] using a kernel function that is tailored to your specific problem. This is also the least intuitive option, so I won't elaborate on that.
Which doesn't answer the current question.