I have been working on a problem whose goal is to find the most accurate coefficient (from a physical point of view) that expresses the relationship between a feature and an output variable. The model is linear, has multiple features, and I am using the normal equation to solve this.
My concern here is that some features are highly correlated, and in my understanding this can lead to some questions about how to find the correct coefficients. I have read that removing these features is sometimes recommended, but in this problem these features are not redundant, i.e. they all contribute to explaining the output variable.
So, how should I proceed to find the correct coefficients without discarding any features?
Edit: I think I should have provided a simple example of the real problem to make this more clear and interesting:
- Consider a small village where electricity consumers are connected to the same grid.
- I would like to know how the voltage at my home is affected by the consumption of consumer A, consumer B, ....
- Part of these consumers have PV panels, which inject power into the grid (it's like having a negative consumption).
- Injecting power affects the voltages of the grid, particularly near that injection point. For example, if my neighbor is injecting power into the grid, the voltage at my home will increase more than if that injection occurred farther.
- During the day, electricity consumption may be low because people are not at home but all the PV panels are injecting power in a very similar way. As you can see bellow, when consumers install PV panels, their consumption patterns become highly correlated. However, I cannot discard any of them because each one is influencing my voltage. Closer to me, higher the influence.