# Dirac delta function for multivariate input (in the context of Gaussian processes)

Let's say we have a set of $$N$$ observations $$D = \{\bf X, t\}$$ where $${\bf X} = [{\bf x}_1, ..., {\bf x}_N]^T$$ are the locations and $${\bf t} = [t_1, ..., t_N]^T$$ are the targets.

When applying a Gaussian process for the problem of regression, we start with an assumption that

$$$$t_i = f_i + \epsilon_i$$$$

where $$t_i$$ is the observed targets, $$f_i$$ is a random variable that is the output at location $${\bf x}_i$$, and $$\epsilon \sim \mathcal{N}(0, \sigma^2)$$ is assumed noise. We are then interested in the joint marginal distribution of $${\bf t}$$, which we find to be

$$$${\bf t} \sim \mathcal{N}({\bf t}|{\bf 0}, {\bf C})$$$$

where the covariance matrix $$\bf C$$ is defined by

$$$${\bf C}_{n,m} = k({\bf x}_n, {\bf x}_m) + \sigma^2\delta_{n,m}$$$$

The book I'm reading this from (Pattern recognition and Machine Learning, Bishop 2006, pp 306) does not explain what $$\delta_{n,m}$$ is, but from this thread I've come to understand that this is the Dirac delta function

$$$$\delta_{n,m} = \delta({\bf x}_n - {\bf x}_m)$$$$

Any definition I've found assumes the input is univariate (one-dimensional), but the book strongly suggests that the input is multivariate.

My question is this: what is the definition of the Dirac delta function for multivariate input?

• I believe this $\delta_{n,m}$ must be the Kronecker delta, not the Dirac delta. – whuber May 14 '19 at 18:59

## 1 Answer

Dirac's delta notation is $$\delta(x)$$. When you see indices used $$\delta_{ij}$$, it must be Kroneker delta. In Bishop's book you can see how these two notation are used. For instance, check Exercise 3.4 for the latter, and Eq.4.146 for the former.

• Just to be clear, the person (Alexey Zaytsev) who referred to this function as the 'Dirac delta function' in the question I linked to was wrong? – Mossmyr May 14 '19 at 20:18
• @Mossmyr, I wanted to say "No", but the way he defines Dirac delta $\delta(x)$ is wrong. I'm going to comment on his answer now. – Aksakal May 14 '19 at 20:26
• @Akasal But Alexey is talking about the same thing as Bishop (pp 306-307): the covariance of the marginal distribution of the target in a Gaussian process. They use different variable names (Alexey uses $y_i = f_i + \epsilon_i$ while Bishop uses $t_i = y_i + \epsilon_i$), but it's the same thing. Edit: I see that you changed your comment now, disregard my comment. – Mossmyr May 14 '19 at 20:30
• @Mossmyr, yes, I think he meant to use Kronecker delta. Dirac delta is not an ordinary function, it's a generalized function and is defined through integrals – Aksakal May 14 '19 at 20:32