# When may the Kernel Trick Matrix be non symmetric?

Ridge Regression can be expressed as $$\hat{y} = (\mathbf{X'X} + a\mathbf{I}_d)^{-1}\mathbf{X}x$$ where $$\hat{y}$$ is the predicted label, $$\mathbf{I}_d$$ the $$d \times d$$ identify matrix, $$\mathbf{x}$$ the object we're trying to find a label for, and $$\mathbf{X}$$ the $$n \times d$$ matrix of $$n$$ objects $$\mathbf{x}_i = (x_{i,1}, ..., x_{i,d})\in \mathbb{R}^d$$ such that:

$$\mathbf{X} = \begin{pmatrix} x_{1,1} & x_{1,2} & \ldots & x_{1,d}\\ x_{2,1} & x_{2,2} & \ldots & x_{2,d}\\ \vdots & \vdots & \ddots & \vdots\\ x_{n,1} & x_{1,2} &\ldots & x_{n,d} \end{pmatrix}$$

We can kernelise this as follows: $$\hat{y} = (\mathbf{\mathcal{K}} + a\mathbf{I}_d)^{-1} \mathbf{k}$$

where $$\mathbf{\mathcal{K}}$$ is the $$n \times n$$ matrix of kernel functions $$K$$

$$\mathcal{K} = \begin{pmatrix} K(\mathbf{x}_1,\mathbf{x}_1) & K(\mathbf{x}_1,\mathbf{x}_2) & \ldots & K(\mathbf{x}_1,\mathbf{x}_n)\\ K(\mathbf{x}_2,\mathbf{x}_1) & K(\mathbf{x}_2,\mathbf{x}_2) & \ldots & K(\mathbf{x}_2,\mathbf{x}_n)\\ \vdots & \vdots & \ddots & \vdots\\ K(\mathbf{x}_n,\mathbf{x}_1) & K(\mathbf{x}_n,\mathbf{x}_2) &\ldots & K(\mathbf{x}_n,\mathbf{x}_n) \end{pmatrix}$$

and $$\mathbf{k}$$ the $$n \times 1$$ column vector of kernel functions $$K$$

$$\mathbf{k} = \begin{pmatrix} K(\mathbf{x}_1,\mathbf{x})\\ K(\mathbf{x}_2,\mathbf{x}) \\ \vdots \\ K(\mathbf{x}_n,\mathbf{x}) \end{pmatrix}$$

What about the case where the training data, $$\textbf{x} \in \mathbb{R}^{40}$$ and $$\textbf{y}$$ is the test data $$\in \mathbb{R}^{10}$$?

But the Kernel, $$\mathcal{K}$$ loses its symmetry matrix? Does this make sense? You take the dot product between training and test data and do not have a a square matrix.

• It is not. The kernel must be symmetric to learn the coefficients, but the functional form of the prediction doesn't require a symmetric K May 14, 2021 at 17:06
• If you had 40 training points, and 10 test points, how do you even define the kernel?
– user318514
May 14, 2021 at 17:07
• A 40x10 matrix. May 14, 2021 at 17:07
• When you do gradient descent, the train and test sizes aren't equal, so you can't learn the coefficients, with unequal train and test sizes?
– user318514
May 14, 2021 at 17:12
• You learn coefficients using only the train data (so a 40x40 matrix in you example) May 14, 2021 at 17:25