# Linear regression: highly-correlated features but not redundant

I have been working on a problem whose goal is to find the most accurate coefficient (from a physical point of view) that expresses the relationship between a feature and an output variable. The model is linear, has multiple features, and I am using the normal equation to solve this.

My concern here is that some features are highly correlated, and in my understanding this can lead to some questions about how to find the correct coefficients. I have read that removing these features is sometimes recommended, but in this problem these features are not redundant, i.e. they all contribute to explaining the output variable.

So, how should I proceed to find the correct coefficients without discarding any features?

Edit: I think I should have provided a simple example of the real problem to make this more clear and interesting:

• Consider a small village where electricity consumers are connected to the same grid.
• I would like to know how the voltage at my home is affected by the consumption of consumer A, consumer B, ....
• Part of these consumers have PV panels, which inject power into the grid (it's like having a negative consumption).
• Injecting power affects the voltages of the grid, particularly near that injection point. For example, if my neighbor is injecting power into the grid, the voltage at my home will increase more than if that injection occurred farther.
• During the day, electricity consumption may be low because people are not at home but all the PV panels are injecting power in a very similar way. As you can see bellow, when consumers install PV panels, their consumption patterns become highly correlated. However, I cannot discard any of them because each one is influencing my voltage. Closer to me, higher the influence.

– Dave
Commented Feb 22, 2022 at 13:12
• @Dave: From what I understood, if two variables are highly correlated it may be difficult to extract the real influence of each one of these feature on the output. For prediction it may not be a problem but in my case that influence has a physical meaning that I need for other purposes. Commented Feb 22, 2022 at 13:36
• Correlated variables means that once you know about one, you know about the other. It would be similar to including a variable twice in the model. You have to either use one or the other or a combination of both such as a sum or a ratio. Commented Feb 22, 2022 at 17:56
– Dave
Commented Feb 22, 2022 at 18:21
• Looks that your model should be [Voltage in my home] ~ Cons[A,B,...] + PV. Commented Feb 23, 2022 at 19:21