If you want to predict a range for a regression problem using a deep network, you can do quantile regression and go with (for example) 5% and 95% quantiles. The other option is predicting a probability density function (using something like tensorflow probability) and use its cdf to get a range. What is the advantage/disadvantage of one versus the other? Are there circumstances that there is a clear choice?

  • $\begingroup$ Isn’t the TensorFlow probability the predicted probability of class membership in a classification problem? $\endgroup$
    – Dave
    Aug 30 '20 at 0:27
  • $\begingroup$ It might be useful for that as well (I don't know tbh) but the context of my question is using tensorflow probability layers (something like this tensorflow.org/probability/api_docs/python/tfp/layers/…) and setting the output as a distribution. $\endgroup$
    – zhino
    Aug 30 '20 at 1:03

There are several ways to issue probabilistic forecasts. Using a predictive density function is the parametric approach. For example, if you assume errors are Gaussian, you only need to estimate their mean and variance in order to calculate prediction intervals. This requires less computation, but you have to make assumptions about the underlying distribution, so it might be inflexible.

Alternatively, you can estimate separate quantiles via quantile regression. If you estimate a large set, say ${0.01, 0.02, .., 0.99}$ you can approximate the probability distribution function. This is considered a non-parametric approach and it's more flexible, since you avoid making any assumptions about the distribution. However, it might require a different model to be trained for each quantile, which increases the computational burden.

There are also other ways to solve this problem, for example bootstrapping is a popular approach. I believe it depends on the problem at hand; if a known distribution fits your problem well, I would follow the parametric approach. In general, I think the non-parametric is more flexible and popular in practical applications (at least in my domain). I have also seen applications where quantile regression is used to estimate the range from $5\%-95\%$ and exponential distributions are used for modelling the tails.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.