New answers tagged

0

I might have figured it out. Since $Z_i(t)$ and $Z_j'(t)$ are zero-mean, $E(Z_i(t)Z_j'(t))=\text{Cov}(Z_i(t),Z_j'(t))$. For $(i,j)\in A$, this quantity is non-negative. Thinking of covariance as an inner product, we see that $Z_i(t)$ can be written as a non-negative multiple of $Z_j'(t)$ plus an orthogonal component $W_{ij}$. In addition, for Gaussian ...


0

Extended comment. In slide 4/6 of the lecture slides which you linked in the comments: In light of this, I am unable to understand how or why your query remains concerning whether the posterior is a Gaussian process? Is the presence of $\mathbf{x}$ and the $\mathcal{M}_i$ on the right hand side preventing you from recognising the $p(f | \mathbf{x}, \mathbf{...


0

I ran some tests and it looks like (regular) GPs are not able to achieve this. As one of the comments above mentioned, GPs may learn $\tilde{f}(x) = f(x) + c$ and $\tilde{g}(z) = g(z) - c$ for any arbitrary constant $c$, so while the sum $f(x) + g(z) = \tilde{f}(x) + \tilde{g}(z)$ can be learned by a GP, the individual summands can't. Parametric methods, ...


2

$\newcommand{\0}{\mathbf 0}$Let $z \sim \mathcal N_n(\0, I)$ and $LL^T=\Sigma$. I'll ignore $\mu$ as I don't think it adds anything here. Here's how we can think of $Lz$ as forming our multivariate Gaussian $X \sim \mathcal N_n(\0, \Sigma)$. My main intuition here is that each new row of $L$ adds in a new independent variable that hasn't appeared in any of ...


0

You can estimate the model parameters $\mu$ & $\tau^2$ using the MLE approach (along with other parameters of the kernel) or simply using the analytical GLS solutions. If you choose MLE, while using gradient-based algorithms (like L-BFGS-B), you need to provide good initialization points for $\mu$ & $\tau^2$, in order to obtain robust estimates. The ...


1

Many common Random processes are Gaussian processes: white noise, random walk etc. Their applications are uncountable. For other downstream tasks there are variants of Gaussian processes: Gaussian processes for sensitivity analysis (based on Gaussian process regression); Unsupervised learning and Deep GP [1] Due to their flexibility and availability of ...


4

While using gradient-based optimizers like L-BFGS-B, you need to ensure that you choose good enough initialization points for the optimizer. This helps in efficient minimization of the NLL and also prevents early stopping. For instance, you can use a grid-search-based approach (recommended in this article) to initialize the lengthscale ($𝐿_𝑖$) values and ...


0

Infinite dimensional Gaussian processes have sample functions which span an infinite dimensional space (subspace of Hilbert space of mean square integrable functions). Equivalently, the kernel expansion requires an infinite number of terms (Mercer's theorem). It is possible to have Gaussian random processes with countable or even uncountable index sets whose ...


1

Suppose we have $X \sim \mathcal N_n(\mu, \Sigma)$. We can think of $X$ as giving us a random function from $\{1, \dots, n\}$ to $\mathbb R$, which we evaluate by indexing so e.g. $X(1) = X_1$. The space of random functions with this domain is $n$-dimensional since it is spanned by the functions $\{e_1, \dots, e_n\}$ where $e_i(t) = \mathbf 1_{t=i}$ are just ...


5

While a sample from a multivariate Gaussian distribution produces a vector with a discrete number of elements, a sample from a Gaussian Process is a continuous function, which is "infinite-dimensional" in the sense that it is "indexed" by a continuously varying coordinate. I'm not an expert in GPs, but I've found this page helpful.


3

The second term is the determinant of the covariance matrix $$ |K + \sigma^2_nI| = \det(K + \sigma^2_nI), $$ which is a polynomial function that outputs a scalar value.


1

That is the exact procedure used in GP. Kernel parameters obtained by maximizing log marginal likelihood. You can use any numerical opt. method you want to obtain kernel parameters, they all have their advantages and disadvantages. I dont think there is closed form solution for parameters though.


0

In my opinion, a good answer is written in Pawitan (2001, p.29) "In All Likelihood: Statistical Modelling and Inference Using Likelihood." It is a better information utilization. You may consider the basic definition of a likelihood function: $$l(\theta;x) = f(x; \theta)$$ However, this implies: $$f(\theta|x) = constant \times f(\theta)l(x|\theta)$$...


0

I would scale/center the features just as in a cluster analysis, e.g. kmeans. The distance between all features is equal after scaling, so no feature would dominate in your kernel/SVM. In theory it should help and it is in general the same process, because in cluster analysis unscaled features face the same problem, because the algorithm would not bother ...


0

The GP typically represents the mean value, which should be deterministic (c.f. a regression line has no "wobble"). You can add a "nugget" term to account for noise in observed data. In this case the covariance is modified to become $$C(x,x') = K(x,x') + \lambda^2 \times I(x == x')$$ Here, $K(x,x')$ is any appropriate covariance function (...


1

The matrices $K_{a,a}$, $K_{b,b}$ are positive definite by construction. Using the Schur complement the joint covariance matrix, which is also positive definite, can be decomposed into: $$ M = {\begin{bmatrix}K_{a,a}&K_{a,b}\\K_{b,a}&K_{b,b}\end{bmatrix}}={\begin{bmatrix}I&K_{a,b}K_{b,b}^{-1}\\0&I\end{bmatrix}}{\begin{bmatrix}K_{a,a}-K_{a,b}...


Top 50 recent answers are included