One assumption for regression analysis is that $X$ and $Y$ are not intertwined. However when I think about it It seems to me that it makes sense.
Here is an example. If we have a test with 3 sections (A B and C). The overall test score is equal to the sum of individual scores for the 3 sections. Now it makes sense to say that $X$ can be score in section A and $Y$ the overall test score. Then the linear regression can answer this question: what is the variability in overall test score that is attributable to section A? Here, several scenarios are possible:
- Section A is the hardest of the 3 sections and students always score lowest on it. In such a case, intuitively $R^2$ would be low. Because most of the overall test score would be determined by B and C.
- Section A was very easy for students. In this case also the correlation would not be high. Because students always score 100% of this section and therefore this section tells us nothing about the overall test score.
- Section A has intermmediate difficulty. In this case the correlation would be stronger (but this also depends on the other scores (B and C).
Another example is this: we analyze the total content of a trace element in urine. And we analyze independently the individual species (chemical forms) of that trace element in urine. There can be many chemical forms. And if our analyses are correct, the sum of chemical forms should give us the same as the total content of an element (analyzed by a different technique). However, it makes sense to ask whether one chemical form is correlated with the total element content in urine, as this total content is an indicator of the total intake from food of that element. Then, if we say that $X$ is the total element in urine and $Y$ is chemical form A in urine then by studying the correlation we can explore whether this chemical form is the major one that contributes to the overall variablity or not.
it seems to me that it makes sense sometimes even when $X$ and $Y$ are not independent and that this can in some cases help answer scientific questions.
Would you think $R^2$ can be useful or meaningful in the examples above ? If we consider the test score example above, I would already say there would be about 33% contribution of each section had the difficulty been exactly the same for the students. But in practice this is not necessarily true. So I was thinking maybe using regression analysis can help us know the true variability attributed to each section of an exam. So it seems to me that $R^2$ would be meaningful even though we already know the null hypothesis is not true.
Are there alternative modified regression methods to account for such situations and provide us with meaningful parameters?