I have two data sets collected by different methods about the same phenomena, one 'bare' and other where I believe there is a constant value added (the same phenomena, with the data collected by a different method, plus a constant percentage value added to each data point). I´m trying to calculate

  1. Check if the two data sets are really representing the same phenomena and
  2. The constant percentage difference.

I made two linear regressions with statsmodels on each data set and can see that the regression lines are parallel. I was thinking on

  1. Calculate the correlation of the two data sets to answer 1 - seems to be ok
  2. Subtracting the difference between the two regression lines to evaluate the percentage difference, but it seems to be incorrect to do so.

How can I accomplish this?

  • $\begingroup$ Parallel regression lines would suggest that a constant had been added to the values, rather than a constant percentage applied (which would be understood as multiplying each value $y$ by a constant $1 + \delta/100$, where $\delta$ is the percentage). Could you clarify which of these operations you mean? And exactly what "linear regressions" are you performing? Is there another variable involved? $\endgroup$ – whuber Aug 7 '15 at 13:29
  • $\begingroup$ I´m doing a linear regression with only one dependent variable. Having the same phenomena and two different ways of gathering the data, I agree that I have two sets of variables. There is no other variable involved. $\endgroup$ – Ivan Aug 12 '15 at 12:13
  • $\begingroup$ If you gather the data in two different ways, then you have two variables, not one (or, if you like, two distinct measurements of the same "variable"--but those measurements must be managed and analyzed as two separate variables). But is there some other variable involved in your regressions or are you regressing these variables against each other? $\endgroup$ – whuber Aug 12 '15 at 12:16
  • $\begingroup$ I have the same independent variable for both experiments, I´m not regressing the two dependent variables against each other. $\endgroup$ – Ivan Aug 12 '15 at 12:18
  • $\begingroup$ Thanks again, @whuber, you clarified a conceptual problem. It is indeed a $1+\delta/100$ thing, not a constant value, so indeed the two regression lines should not be parallel. The comparison and finding the $\delta$ value problems stand. $\endgroup$ – Ivan Aug 12 '15 at 12:47

Gregory Chow developed https://en.wikipedia.org/wiki/Chow_test to statistically compare two regression equations. If the scale of the dependent variable is different then you might consider normalizing the Y values before the analysis. Another approach is to study each data set for a model that might include lags and/or deterministic structure and perhaps a power trasndormation. Careful analysis/review of the two models might suggest the nature of the differences, if any.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.