3
$\begingroup$

I'm investigating how some weather variables (15) affect electricity demand in a specific area during the last 20 years. I was thinking to perform the following steps: 1. Perform Multiple Linear Regression on each subset of selected variables 2. Save t-statistics (p-values) for each run

Then, I would to show the statistics (median, min, max, quantiles) of the t-statistics for each variable in order to give an idea about which is the most influencing. Finally, I would also show the relationship between each variable and the mean square error obtained with regressions using it.

Do you think this approach makes sense?

$\endgroup$
4
$\begingroup$

No. There are many variable selection techniques including stepwise, forward and backward selection or if you have only a small number comparing all subsets may be feasible. But the model selection criterion should be something like AIC, BIC or Mallow's $C_p$. These are the most common approaches in the literature.

$\endgroup$
  • $\begingroup$ I'm not looking for the best model but I just want to analyze the influence of single variables. $\endgroup$ – Matteo De Felice Jul 16 '12 at 11:25
  • $\begingroup$ Ok, so I won't compare t-statistics and I'll calculate AIC or Mallow's CP for each subset and then I'll evaluate which variable leads to better performances. $\endgroup$ – Matteo De Felice Jul 16 '12 at 12:06
  • 1
    $\begingroup$ Adjusted R-square is another measure you can enter into the mix. For each variable the percentage of variance it explains given the other variables in the model would be an intuitive measure of the variable's importance. This would be a partial R-square. $\endgroup$ – Michael R. Chernick Jul 16 '12 at 13:04
4
$\begingroup$

It sounds like you have time series data, in which case multiple linear regression is going to give you biased estimates.

Even without that, however, I don't think your approach is correct. In your comment to @Michael you say you want to "analyze the influence of single variables" but influence on what? Presumably on the dependent variable (electricity usage). Then decide: Controlling for what? And include those variables in your model.

And I would make that decision of which variables to include based on substantive knowledge.

If you want to include multiple models, there are methods for model averaging, but they are more complex that what you outline.

$\endgroup$
  • 1
    $\begingroup$ (+1) "I would make that decision of which variables to include based on substantive knowledge." this is good advice that people often decline in favor of some automatic method. $\endgroup$ – Macro Jul 16 '12 at 12:21
  • 1
    $\begingroup$ Amodel including time as a covariate or autoregressive terms along with the covariates might lead to a better model and the importance of a variable by any of the measures mentioned will usually depend on the other variables included in the model. $\endgroup$ – Michael R. Chernick Jul 16 '12 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.