# Tag Info

### Question of understanding regarding Bayesian Optimization, Gaussian process and acquisition function

You are correct. The Gaussian process is a distribution over functions. As with other Bayesian methods, you start with a prior and combine it with data (observed outcome) through likelihood to get a ...
Accepted

### Question of understanding regarding Bayesian Optimization, Gaussian process and acquisition function

In addition to Tim's answer, here are some slight nitpicks/clarifications which might assist your intuitive understanding/prevent possible confusion in the future: We start with a a-prior function, ...

### Marginalization over the nuisance variable

Though, there were great answers, specially from @gunes. The most generic case where there is no independence or conditional independence assumption, marginalising $\mathbf{f}$, over $\mathbf{u}$ ...

### Marginalization over the nuisance variable

As mentioned in the comments, the first multiplicand should be $p(y|u)$ because it's originally $p(y|f,u)$ and it's stated that $y$ and $f$ are conditionally independent given $u$. For the integral, ...

### Theoretical Speculations as to Why Neural Networks have Replaced Kernel-Based Methods

First of all, not only neural networks are universal approximators. There is nothing special about them, they just proved to work quite well for a class of problems. Kernel based methods generally don'...
Accepted

Accepted

### How to derive an inverse of Gaussian Kernel

This would be hard, at least for these commonly used kernels like Matern, squared exponential etc. Suppose that you can obtain an analytical solution of your $K^{-1}(x_i, x_i)$. Then GP regression ...

### Theoretical Speculations as to Why Neural Networks have Replaced Kernel-Based Methods

Your question is more theoretically founded, but goes in the same direction as this one. You might want to check the answers there (disclaimer: one of them is mine). In order to answer your question ...

### Looking for a Picture that shows Statistical Models "Learning" from Data

It's easy to make your own. I'll do it in R. ...

### Why Gaussian process has marginalisation/consistency property?

It is actually a good question which shows a subtlety of the definition of a general(not necessarily Gaussian) stochastic process. And I hope it is not too late for you. In GPML, it says A stochastic ...

### Hyperparameter tuning in Gaussian Process Regression

If solving the linear problem $K\pmb{\alpha} = \textbf{y}$ is too expensive for you at each step of your optimisation, you could resort to approximation techniques such as the Nystr$\ddot{o}$m method (...
Accepted

### What is the entropy of a Gaussian Process?

For stochastic processes, the usual generalisation of entropy is the entropy rate. See here: https://jsri.srtc.ac.ir/article-1-24-en.pdf [in case the link is broken in the future, look up "The ...