Handling opposite correlations at different time resolutions

Question

Here is an interesting phenomenon: solar panel generation at a daily (day of the year) resolution is highly correlated with temperature. This isn't too surprising, summer brings more intense solar irradiance, longer days and generally fewer clouds. However, increased temperature causes the panels to perform less efficiently, meaning that for each bit of light hitting the panel, we get less electricity. This is a direct causal effect. This can be seen more easily at an hourly resolution: warmer days with the same amount of sun generate less energy. Thus, there is an inverse correlation between temperature and generation.

Is there a name for this phenomenon or an area of statistics related to this?

Answer 1

This is a pretty straightforward example of confounding.

The relationship between temperature and output (i.e., due to the temperature effect on panel efficiency) is being confounded by the relationship between season and output. Because season also has a relationship to temperature, failing to control for the confounding variable(s) creates bias in your estimate of the effect you are trying to measure. In this case the bias is so severe that it reverses the direction of the effect.

The solution is that you must control for the confounding variable. There are a lot of ways you could go about that, ranging from simple, like a monthly random intercept, to very complex, like using a simulation based on historical meteorological data to compute the total daily irradiance for each day in your dataset (which you would then control for). What approach you choose depends on the data available to you, the precision needed to answer your research question, and so on. But whatever approach you choose, it starts with recognizing the existence of confounders.

Answer 2

I'd say that this was an example of the ecological fallacy, which occurs when a relationship with a variable at one resolution (typically people) doesn't have the same relationship at another level of resolution (some sort of groups).

A common example cited in the wikipedia article (and there's a whole book about this is that in the US rich people tend to vote Republican, but people in more affluent states vote democrat.

This is often found in epidemiological research. Something that is true at the country level may not be true at the individual level. Countries that eat more sugar have higher levels of certain kinds of cancer, but that doesn't mean that people who eat more sugar are more likely to have cancer. (More affluent countries have higher sugar consumption, and an older population, so they have more cancer.)

Similarly, vaccination against coronavirus decreases your chance of dying of covid-19. Counties with older populations might have more people getting vaccinated (because older people are more concerned about covid) and more deaths due to covid (because people are older), so the relationship is negative at the individual level, and positive at a higher level of aggregation.

Skin cancer is caused (to a large extent) by sun exposure. People who live in countries with a lot of sun are likely to be more aware and more careful about this than people from less sunny countries. The incidence of skin cancer is higher in Norway (age standardized rate: 18.8) and Denmark (age standardized rate 19.2) than Israel (age standardized rate: 11.4) and Italy (age standardized rate 11.4). https://www.ncbi.nlm.nih.gov/books/NBK481862/table/chapter1.t1/

Answer 3

If you have a dataset containing the three features: 1) generated power 2) temperature 3) length of day you can control for the influence of feature 3 by computing the partial correlation coefficient $r_{1,2| 3}$ which should be closer to zero, although $r_{1,2}$ is high. If you do not have feature 3, then this is a confounder: https://en.m.wikipedia.org/wiki/Partial_correlation

$\rho_{XY| Z}= \frac{\rho_{XY}-\rho_{XZ}\rho_{ZY}} {\sqrt{1-\rho_{XZ}^2}\sqrt{1-\rho_{ZY}^2}}$

You can make the same argument with Features 1) foot size 2) number of words in the vocabulary 3) age of a person: Babies with small feet have a small vocabulary and people with bigger feet have a bigger vocabulary (i.e. there will be a big correlation). However, controlling for age, i.e. looking at groups of same age, there will be close to no correlation between foot size and size of the vocabulary.

PS: if the relationship is nonlinear you are better off computing the partial correlation coefficient using Spearman's correlation