0
$\begingroup$

Let's say we have a set of $N$ observations $D = \{\bf X, t\}$ where ${\bf X} = [{\bf x}_1, ..., {\bf x}_N]^T$ are the locations and ${\bf t} = [t_1, ..., t_N]^T$ are the targets.

When applying a Gaussian process for the problem of regression, we start with an assumption that

\begin{equation} t_i = f_i + \epsilon_i \end{equation}

where $t_i$ is the observed targets, $f_i$ is a random variable that is the output at location ${\bf x}_i$, and $\epsilon \sim \mathcal{N}(0, \sigma^2)$ is assumed noise. We are then interested in the joint marginal distribution of ${\bf t}$, which we find to be

\begin{equation} {\bf t} \sim \mathcal{N}({\bf t}|{\bf 0}, {\bf C}) \end{equation}

where the covariance matrix $\bf C$ is defined by

\begin{equation} {\bf C}_{n,m} = k({\bf x}_n, {\bf x}_m) + \sigma^2\delta_{n,m} \end{equation}


The book I'm reading this from (Pattern recognition and Machine Learning, Bishop 2006, pp 306) does not explain what $\delta_{n,m}$ is, but from this thread I've come to understand that this is the Dirac delta function

\begin{equation} \delta_{n,m} = \delta({\bf x}_n - {\bf x}_m) \end{equation}

Any definition I've found assumes the input is univariate (one-dimensional), but the book strongly suggests that the input is multivariate.

My question is this: what is the definition of the Dirac delta function for multivariate input?

$\endgroup$
  • $\begingroup$ I believe this $\delta_{n,m}$ must be the Kronecker delta, not the Dirac delta. $\endgroup$ – whuber May 14 at 18:59
2
$\begingroup$

Dirac's delta notation is $\delta(x)$. When you see indices used $\delta_{ij}$, it must be Kroneker delta. In Bishop's book you can see how these two notation are used. For instance, check Exercise 3.4 for the latter, and Eq.4.146 for the former.

$\endgroup$
  • $\begingroup$ Just to be clear, the person (Alexey Zaytsev) who referred to this function as the 'Dirac delta function' in the question I linked to was wrong? $\endgroup$ – Mossmyr May 14 at 20:18
  • $\begingroup$ @Mossmyr, I wanted to say "No", but the way he defines Dirac delta $\delta(x)$ is wrong. I'm going to comment on his answer now. $\endgroup$ – Aksakal May 14 at 20:26
  • $\begingroup$ @Akasal But Alexey is talking about the same thing as Bishop (pp 306-307): the covariance of the marginal distribution of the target in a Gaussian process. They use different variable names (Alexey uses $y_i = f_i + \epsilon_i$ while Bishop uses $t_i = y_i + \epsilon_i$), but it's the same thing. Edit: I see that you changed your comment now, disregard my comment. $\endgroup$ – Mossmyr May 14 at 20:30
  • $\begingroup$ @Mossmyr, yes, I think he meant to use Kronecker delta. Dirac delta is not an ordinary function, it's a generalized function and is defined through integrals $\endgroup$ – Aksakal May 14 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.