# Dirac delta function for multivariate input (in the context of Gaussian processes)

Let's say we have a set of $$N$$ observations $$D = \{\bf X, t\}$$ where $${\bf X} = [{\bf x}_1, ..., {\bf x}_N]^T$$ are the locations and $${\bf t} = [t_1, ..., t_N]^T$$ are the targets.

When applying a Gaussian process for the problem of regression, we start with an assumption that

$$\begin{equation} t_i = f_i + \epsilon_i \end{equation}$$

where $$t_i$$ is the observed targets, $$f_i$$ is a random variable that is the output at location $${\bf x}_i$$, and $$\epsilon \sim \mathcal{N}(0, \sigma^2)$$ is assumed noise. We are then interested in the joint marginal distribution of $${\bf t}$$, which we find to be

$$\begin{equation} {\bf t} \sim \mathcal{N}({\bf t}|{\bf 0}, {\bf C}) \end{equation}$$

where the covariance matrix $$\bf C$$ is defined by

$$\begin{equation} {\bf C}_{n,m} = k({\bf x}_n, {\bf x}_m) + \sigma^2\delta_{n,m} \end{equation}$$

The book I'm reading this from (Pattern recognition and Machine Learning, Bishop 2006, pp 306) does not explain what $$\delta_{n,m}$$ is, but from this thread I've come to understand that this is the Dirac delta function

$$\begin{equation} \delta_{n,m} = \delta({\bf x}_n - {\bf x}_m) \end{equation}$$

Any definition I've found assumes the input is univariate (one-dimensional), but the book strongly suggests that the input is multivariate.

My question is this: what is the definition of the Dirac delta function for multivariate input?

• I believe this $\delta_{n,m}$ must be the Kronecker delta, not the Dirac delta. – whuber May 14 at 18:59

Dirac's delta notation is $$\delta(x)$$. When you see indices used $$\delta_{ij}$$, it must be Kroneker delta. In Bishop's book you can see how these two notation are used. For instance, check Exercise 3.4 for the latter, and Eq.4.146 for the former.
• @Mossmyr, I wanted to say "No", but the way he defines Dirac delta $\delta(x)$ is wrong. I'm going to comment on his answer now. – Aksakal May 14 at 20:26
• @Akasal But Alexey is talking about the same thing as Bishop (pp 306-307): the covariance of the marginal distribution of the target in a Gaussian process. They use different variable names (Alexey uses $y_i = f_i + \epsilon_i$ while Bishop uses $t_i = y_i + \epsilon_i$), but it's the same thing. Edit: I see that you changed your comment now, disregard my comment. – Mossmyr May 14 at 20:30