0
$\begingroup$

Let's say I have the following regression problem:

Given a person's age and height, I want to predict how many years they've spent playing basketball. However, instead of just regressing on these features, I'd like to actually predict a distribution that I can simulate from: for each sample, I'd like to predict a distribution (for instance, predict the parameters for the normal distribution) for each specific test sample, that I can then draw samples from to simulate on.

More explicit example: after training my model, if I have a test sample with [age=25, height=72"], I'd like to predict the distribution for how many years that person has spent playing basketball, so I can draw samples from that predicted distribution for simulation purposes. Seems similar to quantile regression but I'd like to predict an explicit distribution that I can sample from...

Any tips for how to go about solving this?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

You can do this with linear regression! Whether you use the MLE or minimize the mean squared error, you are actually modeling the mean response, conditional on the predictors, and you are implicitly assuming that the response itself is normally distributed around that mean. For other distributions, you would use a generalized linear model.

What you're looking for also arises naturally as the "posterior predictive distribution" in a Bayesian model. It might be helpful to think of maximum-likelihood-based inference as Bayesian inference with flat priors.

Fully nonparametric density estimation is very difficult in general except in low-dimensional cases. I asked an involved question about it about a year ago, but I can't find it now.

Improve this answer
$\endgroup$
2
  • $\begingroup$ awesome, thank you! for linear regression, would I just use the minimized MSE across the training set to find sigma for the normal distribution? or is there a more clever way to figure this out? $\endgroup$
    – defcal
    Oct 28, 2015 at 0:52
  • $\begingroup$ @defcal almost. It's the sum of squared errors divided by $n-2$ (rather than just $n$). $\endgroup$
    – shadowtalker
    Oct 28, 2015 at 2:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.