2
$\begingroup$

After training a model on the resulting value given some set of features, I would like the model to estimate the probability distribution of the resulting value. I am not interested in classification, I would like the target value to be continuous.

For example, I might have S&P day close, S&P day open, trade volume, day high, and day low as features and a target being tomorrows day open. I would like the model to learn the probability distribution of tomorrows day open given these features. Be it through representing the parameters of the distribution, or being able to evaluate the probability of a feature set resulting in a specific target value.

Bonus points if this technique can be applied to a multi-target system.

Edit:

So I found a few methods, as below mentioned, there is Gaussian Process Regression. I found a very well documented implementation in scikitlearn

I also found Students-T Process Regression which is useful for me because the distribution I was fitting had a slight skew and a fat tail, though I couldn't find any implementations of it anywhere.

The most useful solution I have found so far is Mixture Density Networks which allows you to take a set of features and create a mixture distribution which isn't limited to normal.

I also found an RNN Based mixed density network- http://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/

$\endgroup$
1
  • $\begingroup$ Hi there. I have this exact question, and am considering a variety of options as you did. Curious as to what you found in your research, and what ended up being useful to you? The "multi-target system", aka estimating the full joint distribution, is the end goal for me as well for risk analysis, however I'll settle for single-variable marginal distributions (distribution of a single asset's return) which can be stitched together with some sort of "best fitting" copula. $\endgroup$
    – JoseOrtiz3
    Dec 3, 2018 at 20:49

1 Answer 1

1
$\begingroup$

The example problem you gave sounds like a regression problem, since tomorrow's day open is a continuous value you'd like to predict.

A typical approach would be to assume a distribution for your target value and estimate its parameters given some input. There are many ways to do this, but Gaussian process regression might be a good fit. Gaussian processes produce an estimate of the mean and variance of the target variable, which together specify the probability distribution. They can be quite flexible for many applications.

Here is a good tutorial: https://www.robots.ox.ac.uk/~mebden/reports/GPtutorial.pdf

And "thee book" with a link to some GP software: http://www.gaussianprocess.org/gpml/

$\endgroup$
2
  • 1
    $\begingroup$ This is a very interesting method, I've spent the day looking over it. Do you know whether this technique can be applied to other distributions? Otherwise, I think my best option is to attempt to transform the distribution of the data I'm looking at to normal and make assumptions about skew and kurtosis. $\endgroup$
    – user120010
    Nov 10, 2016 at 23:51
  • $\begingroup$ @scherm I have to solve a problem. take a synthetic probability distribution, or a data set e.g. s&p 500 get the generator, not the probability distribution do this using classical ML, not using quantum ML find minimum of KL-divergence loss function, using scipy, not tensorflow/keras Does this seem doable? I am stuck on step 2... $\endgroup$ May 17, 2021 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.