2
$\begingroup$

I am trying to develop a benchmark method for energy efficiency analysis. The idea is to predict energy consumption and use the difference between predicted and real energy consumption as a measures of inefficiency.

I developed two model that predict energy consumption using the same explanatory variables and I have to choose the best. The only difference in the two model is the dependent variable. In one case Y (say Y1) is total energy consumption and in the other one Y (say Y2) is energy intensity (kWh/m2).

As I said the two model are identical, but model 1 has a much higher adj. R2 (0.95), while the second has adj. R2 = 0.7. What I can say is that SIZE is one of the explanatory variable and has a huge impact on the first model explaining the majority of the variability on energy consumption. In the other case (model 2), SIZE has a much smaller impact comparable with the other explanatory variables.

My question is which one should I choose? The first one? The higher R2 is useful for my objective?

thanks, Stefano

$\endgroup$
4
  • $\begingroup$ I doubt that the energy intensity is normally distributed. So maybe the assumptions for linear regression are not met. Also have a look at index composition methods, which seems to be a common benchmarking method in the energy efficiency field. $\endgroup$
    – Karsten W.
    Feb 1, 2017 at 21:48
  • $\begingroup$ Another option wouldo be to look at data envelopment analysis, which is common in benchmarking and becomes more and more popular in energy efficiency IMO. $\endgroup$
    – Karsten W.
    Feb 1, 2017 at 22:23
  • $\begingroup$ I forgot to say that functional form is in the log-log function: lnY=β_(o )+ β_(1 ) lnX+ ε $\endgroup$
    – Ste
    Feb 2, 2017 at 14:24
  • $\begingroup$ I have already used DEA in an order study. Although I find it useful in some cases now I am trying to find variables that have impact on energy consumption, and regression analysis is much more informative since coefficients of the regression can be used to determine the impact of independent variables on dependent variable (energy consumption). $\endgroup$
    – Ste
    Feb 2, 2017 at 14:32

1 Answer 1

2
$\begingroup$

So as I think about your problem, I see two issues. The second issue is which model is better given the same independent variables. The first issue is the theoretical soundness of doing that.

Measurements like $R^2$ are irrelevant for this type of question. You need to ignore it. Let us work on the first issue, which is soundness. Y1 is consumption. Y2 is intensity. Consumption over time is fundamentally an integral. It is a series of additions. Intensity is fundamentally a derivative and is basically a set of ratios. If these are your left-hand side of the equations, how is it even remotely possible they are the same on the right-hand side? If they are of the same form, then one of them is theoretically unsound. They are not even in the same units.

When one does a regression model, one hopes to find the model that matches nature.

For the second question, you can calculate the BIC for each. Before you do that, I would graphically map out all my univariate densities and bivariate relationships to see if there are any joint relationships that could make everything else problematic.

I become deeply worried when I see a model with a very high $R^2$ unless I can explain it away because of very obvious mathematical relationships. There is a real danger of over-fitting.

$\endgroup$
5
  • $\begingroup$ Thank you for your answer. I am not really sure to have understood completely your comment. Maybe I was not clear in my question. I said the the two model are identical, I mean that variables of the model are identical (a part for DV). As expected variables have opposite signs, since a variable that have a positive effect on energy consumption (increase energy consumption) in model 1 (DV=total energy consumption) has negative effect on model 2 (DV= energy intensity). $\endgroup$
    – Ste
    Feb 2, 2017 at 14:35
  • $\begingroup$ Also the reason why the first model has R^2 very high is basically due to the fact that variable SIZE has a very big impact on energy consumption. On the other case using kWh/m^2 as dependent variable what I do is normalize energy use on dimension, so variable SIZE lose importance. Even if is always significan in the model, because there is a scale effect: bigger the building lower the energy consumption. $\endgroup$
    – Ste
    Feb 2, 2017 at 14:43
  • $\begingroup$ Anyway my question is: is there any difference between the two model? If my objective is evaluate energy efficiency and energy efficiency is the difference between predicted and actual energy use, which model should I prefer? $\endgroup$
    – Ste
    Feb 2, 2017 at 14:47
  • $\begingroup$ @Ste lets imagine I regressed on Z1 which is measured in total miles and Z2 which is measured in miles per hour. Then the independent variables for Z1 should only be measured in miles. If one of them is naturally in MPH then it should be multiplied by time in the equation. Likewise, if Z2 is in MPH then either all the independent variables should be in MPH or there should be an hours variable added to the entire equation. So log(Z2)=log(a)+log(b)-log(hours) or log(Z2)=log(a+b)-log(hours) and so on. Your models should be intrinsically incompatible once translated into the physical world. $\endgroup$ Feb 2, 2017 at 17:59
  • $\begingroup$ @Ste regression follows theory, not the other way around. Using the AIC or the BIC would be fine, but Y1=f(a,b) versus Y2=f(a,b) does not make sense because they are not in the same unit measure. One of them must be false by design, regardless of the statistical qualities. $\endgroup$ Feb 2, 2017 at 18:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.