1
$\begingroup$

I'm struggling to grasp Gaussian Process Regression (GPR). I am familiar with the weight-based methods since we can use a basic linear regression to understand about how it works.

While I understand weight-based methods like neural networks, I find it challenging to find straightforward explanations or simple examples for GPR. I've come across mathematical explanations but haven't found a basic, easy-to-understand example. Could someone provide an intuitive explanation or an accessible example of how GPR works?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Gaussian processes are a weight-based method. Although the predictions are computed by matrix algebra, this really just boils down to $\hat{y} = \sum w_i y_i$. It's worth noting that linear regression is a special case of GPR $\endgroup$
    – jcken
    Commented Nov 2, 2023 at 11:54

1 Answer 1

Reset to default
0
$\begingroup$

So conceptually, GPR assumes that every data point is its own dimension and this is key. So it uses “covariance functions” to measure how similar any two observations are. Next GPR uses conditional Gaussian distributions to sample covariance functions. This sounds odd but remember that it’s treating every observation as its own dimension the. using a multivariate Gaussian to sample from where the covariance is given by your kernel (ex: radial basis function)

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ The big question is whether the computational burden of GPR is worth it when compared to fast flexible simpler approaches such as regression splines. $\endgroup$
    – Frank Harrell
    Commented Nov 7, 2023 at 13:05
  • $\begingroup$ Big thing with splines if 1/ what if you chose wrong breakpoints? And 2/ unless you explicitly use a Bayesian implementation the model won’t tell you how confident it is or isn’t) $\endgroup$
    – jbuddy_13
    Commented Nov 8, 2023 at 14:40
  • 1
    $\begingroup$ Knots are not breakpoints except on the 3rd derivative (if using cubic splines). Restricted cubic splines (natural splines; linear beyond outer knots) are very insensitive to the choice of knot locations. More here. Since knot locations don't matter much, if you don't have outside information you can put knots where the X-variable data are dense, i.e., you can allow the 3rd derivative (jolt/jerk) to change in data regions where there is high information content to enable estimation of such changes. $\endgroup$
    – Frank Harrell
    Commented Nov 8, 2023 at 15:03
  • $\begingroup$ No kidding- thanks Frank! I might have dismissed splines prematurely $\endgroup$
    – jbuddy_13
    Commented Nov 8, 2023 at 15:20
  • $\begingroup$ Thanks is expressed here with upvoting. Thx $\endgroup$
    – Frank Harrell
    Commented Nov 8, 2023 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.