Probability distribution of Y in regression?

Question

I'm trying to predict the probability distribution of $Y$ given $X_0, X_1, ...$ with a nonlinear regression. The probability distribution of $Y$ is likely not normal.

So far, I've set up and trained a regression model (support vector regression - SVR) which outputs the average value of $Y$, but I'm interested in seeing the entire distribution. What can I do?

(Please forgive me if this question doesn't make sense –– I'm very new to all this and still trying to figure things out. Any pointers would be appreciated!)

Answer 1

One thing you could do is look at quantile regression for different quantiles. That way you could for example predict how the $5^\mathrm{th}$ percentile differs for different values of your $X$s, how the $10^\mathrm{th}$ percentile differs for different values of your $X$s, etc.

Answer 2

If you build your method around semiparametric regression models you can estimate the entire distribution of Y for each X. A good example is the proportional odds ordinal logistic regression model. I have a case study doing this in my RMS book and course notes where at the end you'll see a nomogram displaying all sorts of predicted quantities (mean, quantiles, exceedance probabilities). See the chapter on ordinal models for continuous Y.