I have been hearing this frequently that gaussian processes is a smoothing operation. I didn't get what they mean by that. Any clarifications guys?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
One way to think of gaussian processes is a kernel density estimation with a fixed-finite number of kernels not fixed at the data. In this interpretation, the arguments for why KDEs are smoothing apply.
$\begingroup$
$\endgroup$
From the book Gaussian Processes for Machine Learning by Rasmussen and Williams; If you're doing GP regression, and you want to predict a value at a point $\mathbf{x}^*$, the posterior predictive mean is given by:
\begin{align*} \overline{f}_{*} = \mathbf{k}^T_* (K + \sigma^2_n I)^{-1} \mathbf{y} \end{align*}
where $\mathbf{y}$ is the vector of observed outputs. Note this is a linear combination of the observed values $\mathbf{y}$, that is it can be rewritten as:
\begin{align*} \overline{f}_{*} = \sum_{c =1}^{n} \beta_{c} y^{(c)} \end{align*}
As I understand it using a linear combination of the observed values a your predicted mean is a sort of smoothing.
