When making a predictive model that is predicting a continuous outcome, how does one arrive at the final prediction?

Question

For instance, a prediction of 1 million could be:

1. A weighted average of various predictions. ex. a .5 chance of 2 million, a .5 chance of 0, for an expected value of 1 million; or

2. The prediction could be a distribution centered around 1 million where the 'peak' of the distribution happens to be 1 million; or

3. The model could be determining a range of possible predictions, each with a specific probability, and choosing the single most likely one. ex. a .5 chance of 1 million, a .3 chance of 0, and a .2 chance of 50,000 - 1 million is the most likely, so it makes that prediction.

4. Other?

I'm trying to make a predictive model (in either SAS or SPSS) for data with a high proportion of zeros and continuous data that begins after a certain threshold. (ex anything between 0 and 1000 is considered a 0). I'm not sure if I need to run a second model to predict categorical outcomes (0, not zero) to multiply by the results from the continuous prediction. If the models are doing 1 or 2, I shouldn't need to run a second model to predict categorical outcomes, but if its doing the third one I would have to. Thanks! I'm a bit out of my depth

Answer 1

All that you mentioned and others are possible and have been done.

The most common is probably using the mean (expected value) mainly since the math is easiest that way, so it was doable before modern fast computers, though the others are becoming more of an option with the computing power now available. Using the mean dates back to Gauss (others around the time are also credited). Using the mean is equivalent to finding the predictive model that minimizes the sum of the squared error, i.e. if you take the predictions from the model and subtract them from the true values (in your training data set), square the results and add them up, then the "best" model will be the one to minimize this sum, this predicts the mean.

If you use the median (middle value) then that is equivalent to minimizing the sum of the absolute values of the errors and goes back to Laplace. There are also semi-parametric models used in survival analysis and reliability that can predict a median without requiring a predefined distribution.

If your data is believed to be normally distributed then predicting the mean, median, and mode (peak of the distribution) will all be the same, another reason the mean, being the simplest to compute, is often used.

With Bayesian methods you can fit a model with the response being a full distribution (with the parameters depending on other variables) from which you can make predictions using the mean, median, mode, or other summary of the full distribution.

For categorical outcomes you can use logistic or multinomial regression models (or others) to predict the probability of each category (and take the most likely if you want to use that as the prediction and throw away the rest of the information).

The field of decision theory lets you use different loss functions to decide what the "cost" is for different "wrong" predictions and create models to minimize the expected loss (the above models being special cases).

Which to use depends on what you are trying to do and what question(s) you are trying to answer.