Let's suppose we are attempting to predict traffic accidents on some highway. Traffic Accidents would be the response variable in our data set.

We have TWO competing models:

  1. Two predictor variables: "Time of Day" and "Total Number of Cars Passing Particular Point Per Minute"

  2. Two predictor variables: "Precipitation Level (Weather)" and "Number of Cars in FAST LANE Passing Particular Point Per Minute"

The goal is to select which of these two models appears to do a better job of predicting traffic accidents.

Additionally, we want to ascertain based on these two models, under what circumstances Traffic Accidents are highest and lowest. For instance, in model 1), what time of day, and what total car count per minute.

Question: Which statistical software package would you use? What specific procedure would you use? Why?

(Note: SAS and many other packages are available to use in our offices. SAS is preferred.)

  • 4
    $\begingroup$ This is a pretty broad question that has been discussed on this site quite often - have you done a search of all questions that use the tag 'model-selection'? The accepted answer (given by gung) in this thread stats.stackexchange.com/questions/22902/… may be a good starting point $\endgroup$ – Macro May 17 '12 at 23:05
  • $\begingroup$ Yes, I looked over similar questions, but none are exactly as I've asked. Care to contribute an answer? I'd appreciate it. $\endgroup$ – nkormanik May 17 '12 at 23:24
  • 1
    $\begingroup$ It sounds like you're asking about how to choose the model that gives the best out-of-sample prediction accuracy. For this I would use some form of cross validation - en.wikipedia.org/wiki/Cross-validation_(statistics) - the default choice for me would be leave-one-out cross validation but an argument could be made for other choices. I can't give you exactly what you ask for (so I'm not posting this an answer), since I would write my own R code to do this - perhaps this will point you in the right direction for SAS routines to look up. $\endgroup$ – Macro May 17 '12 at 23:44
  • 1
    $\begingroup$ Macro, thanks. Your answer is, then, you would use R. And the procedure you would use is you'd write your own code. Hopefully someone with SAS expertise will come forward with an answer. $\endgroup$ – nkormanik May 18 '12 at 0:01
  • 1
    $\begingroup$ There are functions that compute the likelihood of all possible (linear) combinations of predictor variables given a dataset. I have used the leaps package for R to do this in the past. $\endgroup$ – gregmacfarlane May 18 '12 at 1:10

This is a linear regression problem. In SAS Proc Reg and Proc glm can be used to solve regrassion problems. If you want to know the "best" set of variables to use put all four into the model and use a subset selection criteria. Options could be forward, backward or stepwise. This is not the specific question that you asked because you only want to compare two specific models. To do that apply proc reg twice. The first time with model 1 and the second time with model 2. Look at the goodness of fit and regression diagnostics and as Macro mention you can cross-validate using a test set. I don't know the specifics of how you do that in SAS.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.