As far as I know, both Gaussian mixtures as well as Gaussian processes can be used for regression. My question is: what is better and why?

The answers might contain theoretic insights, practical experience or reference to further resources.


I would like to know which one is better for approximation of nonlinear regression problems $$Y_i=g(x_i)+V_i$$ where $V_i\sim\mathcal{N}(0,\sigma^2)$ for all considered $i$. The function $g(\cdot)$ is assumed to be continuous. For the simplicity sake, we can admit that $x_i\in\mathbb{R}$. The criterion is the one-leave out mean square error.

The second criterion, perhaps more important. Since I assume that both (GM and GP) have the universal approaximation property, I would be curious which one is less complex (in terms of time and/or memory) in order to obtain given level of error.

Finally, I assume that the new data points $(x_i,y_i)$ can be generated on demand with a constant computational compexity.

  • 1
    $\begingroup$ "Better" for what purpose and context? You might as well ask "which is better--digging with a spade or digging with a fork--and why?" Please, then, edit this question to provide this information that is essential for getting objective, useful answers. $\endgroup$
    – whuber
    Oct 21, 2014 at 18:12

1 Answer 1


To answer your last question, Gaussian process is a discriminative model as opposed to generative. Therefore, you will not be able to model $p(x, y)$ using Gaussian process. Gaussian process models $p(y | x)$ instead. To generate samples $(x_i, y_i)$ you need to work with a generative model such as Gaussian mixture model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.