# What is a natural way to define RKHS over mixed spaces (discrete and continuous)?

It is well known that given a kernel $$k$$ over any space $$\mathcal{X}$$, there is a corresponding RKHS (Reproducing Kernel Hilbert Space) associated with the kernel $$k$$. For example, Radial basis function (RBF) kernel defines an RKHS over continuous input spaces. Similarly, string kernels define RKHS over discrete input spaces. If given a mixed input space $$(x_c, x_d)$$ where $$x_c$$ and $$x_d$$ represents continuous and discrete inputs respectively, can we define an RKHS over this mixed space by combining (by sum or product of) the corresponding kernels defined over the continuous and discrete space. How do we analyze such RKHS? My main problem is that I haven't been able to understand how do we analyze the combination of two completely distinct spaces (discrete and continuous) formally?

Update One concrete question I am looking at is analyzing the universality properties of the combined kernel over mixed input spaces. A kernel is universal if all the possible "smooth" functions on a given space are "contained" in its RKHS. Such analysis exists for both continuous and discrete spaces separately. I am trying to prove universality property for a kernel over mixed inputs. This requires analyzing RKHS over mixed spaces starting by properly defining them.

• What do you mean by "analysing such RKHS"? It were helpful if you could either state some questions you would like to answer or give us other examples of the type of analysis you expect. – g g Aug 5 '20 at 7:42
• Thank for your response. One concrete question I am looking at is analyzing the universality properties of the combined kernel over mixed input spaces. A kernel is universal if all the possible "smooth" functions on a given space are "contained" in its RKHS. Such analysis exists for both continuous and discrete spaces separately. I am trying to prove universality property for a kernel over mixed inputs. This requires analyzing RKHS over mixed spaces starting by properly defining them. – randomprime Aug 5 '20 at 7:57
• This involves direct sums and tensor products of Hilbert spaces. Are you familiar with those concepts and how they relate to sums/products of kernels? – g g Aug 5 '20 at 8:47
• I understand direct sums but not tensor products of Hilbert spaces. Can you please describe and provide any reference in this regard? – randomprime Aug 5 '20 at 16:04

## 1 Answer

You can use the so-called ANOVA kernel construction, using tensor products and direct sums of kernels. If $$k_c$$ is a kernel over a space $$\cal X_c$$ and $$k_d$$ a kernel over $$\cal X_d$$, then $$k_{cd}$$ defined by $$k_{cd}((x_c,x_d),(x_c',x_d')) := k_{c}(x_c,x_c')k_{d}(x_d,x_d') \quad x_c,x_c'\in{\cal X}_c,x_d,x_d'\in{\cal X}_d$$ is a kernel over the product space $$\cal X_c\times \cal X_d$$. Better for most applications is to include "main effect" terms and use $$k_{cd}'$$ defined by $$k_{cd}'((x_c,x_d),(x_c',x_d')) := 1 + k_{c}(x_c,x_c') + k_{d}(x_d,x_d') + k_{c}(x_c,x_c')k_{d}(x_d,x_d')$$ The kernels can be over continuous or discrete spaces.