Regression Gaussian Estimates instead of points

Question

I am using a support vector regression is order to get estimates of a variable y. I want to receive a probability distribution of my estimates and not just point estimates. I want to predict Gaussian Distributions for every point where the mean should be my point and I will also give a variance.

What I am thinking in order to avoid to use methods like Gaussian Processes or Probabilistic SVM is to use a history of N windows and to use the error as variance for my new estimates.

For example the std of the error of the last N windows is 0.5. Then the estimate if my prediction is 5 is N(5,0.5).

Is this approach right?

Answer 1

Support vector regression maximizes the margin containing the points, up to slack. The objective function is thus not a probabilistic one, and anything tailored on top will not give you optimal results.

SVMs are not meant to provide this. The already mentioned Gaussian processes excel at this. Neural networks can be used if a special loss function is chosen.