Multiple linear regression with possibly non-independent explanatory variables

Question

For a given household for which I have many years of historical data, I want to predict the home gas consumption (heating) with a few variables among:

date          gas       min_temp   max_temp  mean_temp  relative_humidity absolute_humidity   other_column
2023-01-01    5.8 m^3   -3.0°C     2.3°C       -1.2°C     79 %              4 g/m^3             ...
2023-01-02    4.8 m^3   2.0°C      4.2°C        2.3°C     82 %              4.5 g/m^3           ...
...

I could do a multiple linear regression for

$$\rm{gas\ consumption} = \beta_0 + \beta_1 \rm{min\ temp} + \beta_2 \rm{max\ temp} + \beta_3 \rm{mean\ temp} + ... + \varepsilon,$$

but since many of these variables are not independent of each other (and maybe nearly colinear), doing a standard multiple linear regression might give bad results (for example with some negative $\beta_i$ where it shouldn't). Which better solution can we use?

PCA + multiple linear regression (PCR) or PLS or something else?

Note: I'd like to avoid using all 3 (min, max, mean) temp, if possible. How can we evaluate the loss if using using only 1 temperture variable (the best fit among the 3) instead of the 3 variables?

Answer 1

While both PCR and partial least squares seem reasonable here, I'd say those are better suited to situations where you have more variables. There are other ways to deal with colinearity, including ridge regression, which removes the requirement that the estimates be unbiased in order to greatly reduce the variance of the estimates (that is one of the main negative consequences of collinearity).

However, you should also note that collineariy is not a problem if your only goal is prediction, which you say is your goal. However, you also say (correctly) that

doing a standard multiple linear regression might give bad results (for example with some negative βi where it shouldn't).

but this is a problem of explanation. That is, the parameter estimates are hard to interpret, substantively, because controlling for the other variables makes it a bit confusing.

As to the lost from using just one of the variables, you can run both regressions and see what happens. You can compare $R^2$ or adjusted $R^2$, or residuals, or whatever you like. My guess would be that excluding any of these might result in more outliers and worse predictions, but that's just a guess.