I would like to know how to use a different length scale in a kernel for each dimension of the input. For instance, take an input of dimension 4, I would like to use 4 different length scales where for each coordinate of the input. That is, take the Gaussian Kernel (see https://www.tensorflow.org/probability/api_docs/python/tfp/math/psd_kernels/ExponentiatedQuadratic) for example, I want to define the kernel $k$ as $k(x,y) = exp(- (\frac{(x_1 - y_1)^2}{\gamma_1^2} + \frac{(x_2 - y_2)^2}{\gamma_2^2}))$ instead of $k(x,y) = exp(- \frac{||x - y||^2}{\gamma^2})$ where $x = (x_1, x_2)$ and $y = (y_1, y_2)$.
3
-
$\begingroup$ Does anything prevent you from initially dividing coordinate $i$ by $\gamma_i$ and proceeding with unit scales? $\endgroup$– whuber ♦Commented Apr 19, 2021 at 15:23
-
1$\begingroup$ Well, actually, it is a very good idea. I will try doing it. Thanks. $\endgroup$– AkusaCommented Apr 19, 2021 at 18:10
-
$\begingroup$ @whuber unfortunately, often code is factored in a way where kernels are created in one place and applied in another. So the hyperparameters of the kernel are not always available to divide coordinates. I'll answer the question below. $\endgroup$– Josh AlbertCommented Jun 29, 2023 at 0:05
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
What you can do is use a feature transformation that scales the coordinates, and use a unit length scale.
# Define your length scales.
# For example, for a 3-dimensional feature vector:
length_scale = jnp.asarray([1.0, 2.0, 3.0])
# Create the kernel.
base_kernel = tfp.math.psd_kernels.MaternOneHalf(amplitude=1.0, length_scale=None) # None <=> 1
kernel = tfp.math.psd_kernels.FeatureTransformed(base_kernel,
transformation_fn=lambda x, _1, _2: x / length_scale)
X = jnp.ones((4, 3))
K = kernel.matrix(X, X)
print(K.shape) # (4, 4)
