4
$\begingroup$

I was wondering if anyone could help me clear up the difference between descriptive and predictive modelling. I am trying to build a model to predict where house prices will go up. To do this I need to examine historical data to identify predictors of house price increase, and how they relate to predict an increase.

To me this seems like it fits the description of descriptive modelling and predictive modelling. I am looking at historical data and trying to find the set of rules that summarise how we get from the variables to the current house price, so that I can use the same rules to predict from current conditions to future unknown house prices.

However, this also seems like predictive modelling because we are finding rules to predict an outcome.

So my question is, what is this: descriptive or predictive? And how can I determine whether something is descriptive or predictive modelling?

$\endgroup$
1
  • $\begingroup$ hi, don't you want to point out the answer? $\endgroup$ – nickolay Feb 29 '20 at 18:22
13
$\begingroup$

The best graphic I've seen summarizing this is from here:

enter image description here

The bigger focus is on whether or not the statistical model you're building is for statistical inference, or predictive accuracy. Breiman has a good paper explaining the differences, Statistical Modeling - A Tale of Two Cultures. In Breiman's paper, "data modeling" is equivalent to models built for statistical inference, whereas "algorithmic modeling" is closer to models built for predictive accuracy.

In the comment below, @boscovich mentions G. Shmueli's paper, To Explain or Predict? The paper looks to speak directly to your original question (Sections 1.2 and 1.3), and offers a trove of information beyond, for example: model evaluation and selection is also covered (Section 2.6), and two examples are provided (Section 3). This paper is probably a much more digestible starting point than Breiman's.

The biggest difference IMO is related to model fit and assessment. With models built for statistical inference, you are looking at in-sample fit (i.e. the entire sample population). With models built for predictive accuracy, you are looking at out-of-sample fit (i.e. the dataset which represents your sample population is split into a training and test set, and you judge the predictive accuracy by a measure of error - such as MSE, RMSE, MASE, etc. - on the test set). Rob Hyndman has a good paper covering measures of error, Another Look at Measures of Forecast Accuracy.

One thing that gets a bit confusing is that the terms in-sample fit and out-of-sample fit are often relative terms. Suppose your goal is a statistical model built for predictive accuracy. You randomly sample your dataset, splitting it as a 70/30 training-test set. Your model is built on the 70% training set, and scored (judged) by performance on the 30% test set. The training-test set approach is used to avoid overfitting or misspecification. Ultimately, you may want to deploy your model to the population, or even to a validation set. These can also be considered "out-of-sample". The validation set is data that are (read: should be) from the population, but that you never see during the model building process (either in training or test; for all you know it may not exist), but ultimately one where your model is deployed and scored on. A validation set approach is often used in Kaggle competitions. You build your model on the training set, score on the test set, and then it is scored again on the validation set.

Both model types are making "predictions" but it is the domain of which those predictions are judged that make a difference. For statistical inference, the fit is assessed by examining residuals, p-values at specific levels of alpha, etc. For predictive accuracy, you tend to be less concerned with those, and are more aligned with answering the question "Does this model make accurate predictions out-of-sample?"

$\endgroup$
4
  • $\begingroup$ Maybe you can edit the Breiman paper into your answer, now that you have more points. projecteuclid.org/download/pdf_1/euclid.ss/1009213726 $\endgroup$ – Mark L. Stone Jul 23 '16 at 16:34
  • $\begingroup$ @MarkL.Stone, thank you! I just added it in and expanded on the answer. Greatly appreciated. $\endgroup$ – J.M. Jul 23 '16 at 16:36
  • 2
    $\begingroup$ This is another excellent paper on this subject: G.Shmueli, To Explain or to Predict?, Statistical Science 2010 stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf $\endgroup$ – boscovich Jul 23 '16 at 19:55
  • $\begingroup$ @boscovich thank you! I revised the reply and added the paper - it looks to be a much more digestible starting point than Breiman's. $\endgroup$ – J.M. Jul 23 '16 at 20:10
2
$\begingroup$

You are doing predictive modeling. You know that you're doing predictive modeling when you don't care much (or at all) about how your model is arriving at its predictions as long as its predictions are as accurate as possible without overfitting. Easy examples of this include using any of the following models: gradient boosted trees, random forests, or neural networks.

These models are so complex that even though you could look at their weights and splits if you really wanted to, the amount of information would be too overwhelming to make sense of. It's straight forward enough to make sense of one decision tree but not thousands. The same can be said of neural networks, particularly deep learning for object detection in images. A deep neural network can have millions of weights.

$\endgroup$
0
$\begingroup$
  1. Descriptive Analytics, which use data aggregation and data mining to provide insight into the past and answer: “What has happened?”

  2. Predictive Analytics, which use statistical models and forecasts techniques to understand the future and answer: “What could happen?”

  3. Prescriptive Analytics, which use optimization and simulation algorithms to advice on possible outcomes and answer: “What should we do?”

$\endgroup$
1
  • $\begingroup$ This is a somewhat narrow answer; the question could be answered in much broader terms. It would also help the credibility if a quote or source could be included. Additionally it is a very old question that did not seem to generate much interest. $\endgroup$ – cherub Dec 13 '18 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.