4
$\begingroup$

I'm investigating how some weather variables (15) affect electricity demand in a specific area during the last 20 years. I was thinking to perform the following steps: 1. Perform Multiple Linear Regression on each subset of selected variables 2. Save t-statistics (p-values) for each run

Then, I would to show the statistics (median, min, max, quantiles) of the t-statistics for each variable in order to give an idea about which is the most influencing. Finally, I would also show the relationship between each variable and the mean square error obtained with regressions using it.

Do you think this approach makes sense?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

No. There are many variable selection techniques including stepwise, forward and backward selection or if you have only a small number comparing all subsets may be feasible. But the model selection criterion should be something like AIC, BIC or Mallow's $C_p$. These are the most common approaches in the literature.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I'm not looking for the best model but I just want to analyze the influence of single variables. $\endgroup$
    – Matteo De Felice
    Commented Jul 16, 2012 at 11:25
  • $\begingroup$ Ok, so I won't compare t-statistics and I'll calculate AIC or Mallow's CP for each subset and then I'll evaluate which variable leads to better performances. $\endgroup$
    – Matteo De Felice
    Commented Jul 16, 2012 at 12:06
  • 1
    $\begingroup$ Adjusted R-square is another measure you can enter into the mix. For each variable the percentage of variance it explains given the other variables in the model would be an intuitive measure of the variable's importance. This would be a partial R-square. $\endgroup$
    – Michael R. Chernick
    Commented Jul 16, 2012 at 13:04
4
$\begingroup$

It sounds like you have time series data, in which case multiple linear regression is going to give you biased estimates.

Even without that, however, I don't think your approach is correct. In your comment to @Michael you say you want to "analyze the influence of single variables" but influence on what? Presumably on the dependent variable (electricity usage). Then decide: Controlling for what? And include those variables in your model.

And I would make that decision of which variables to include based on substantive knowledge.

If you want to include multiple models, there are methods for model averaging, but they are more complex that what you outline.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ (+1) "I would make that decision of which variables to include based on substantive knowledge." this is good advice that people often decline in favor of some automatic method. $\endgroup$
    – Macro
    Commented Jul 16, 2012 at 12:21
  • 1
    $\begingroup$ Amodel including time as a covariate or autoregressive terms along with the covariates might lead to a better model and the importance of a variable by any of the measures mentioned will usually depend on the other variables included in the model. $\endgroup$
    – Michael R. Chernick
    Commented Jul 16, 2012 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.