0
$\begingroup$

I am performing a t-test to compare two slopes, each fit to independent X,Y data. Does the nature of this test change depending on the range of slopes being compared? For example, is comparing a slope of 10 to a slope of 1 different than comparing a slope of 0.01 to 0.001, even though both tests compares slopes that are an order of magnitude different?

My calculations follow the math outlined here (http://www.real-statistics.com/regression/hypothesis-testing-significance-regression-line-slope/comparing-slopes-two-independent-samples/) and described in [P. Cohen, S. West and L. Aiken, ``Applied multiple regression/correlation analysis for the behavioral sciences'', Routledge Press, 2014.]

To demonstrate my concern, one can plug in values to this online calculator: https://www.danielsoper.com/statcalc/calculator.aspx?id=103. Fix both sample sizes to 100 and both standard errors to 0.005. If slope 1 is set to 0.1 and slope 2 set to 0.01, the t-test returns (t=12.7, p=0.0) indicating the slopes are different. If slope 1 is set to 0.001 and slope 2 is set to 0.0001, the t-test returns (t=0.13, p=0.9) indicating the slopes are the same. In both cases the slopes are an order of magnitude different, and the amount of data and standard errors are the same, so why does the t-test result change so much?

EDIT: I have learned that the problem is that this test compares the difference between two slopes rather than their ratio. If slope1 is 1.0 and slope2 is 0.1, the difference is 0.9 units. If slope3 is 0.001 and slope4 is 0.0001, the difference is 0.0009 units. Even though both pairs of slopes are an order of magnitude different, the difference between the pairs of slopes is very different. This is why the test changes depending on the range of slopes. I have not yet found a better test, but I recommend anyone else finding this question avoid using the above test unless they are specifically comparing slopes with values near 1.

$\endgroup$
  • 1
    $\begingroup$ Be careful when you read the first linked webpage. $\endgroup$ – user158565 May 25 at 0:28
  • 1
    $\begingroup$ The formula for the t-statistic for the difference in slopes is correct in the first link. But in your last paragraph, you are only changing the value of the numerator in that formula by changing the values of the slopes and are expecting that the formula will return the same result. If you are changing the value of the numerator, the denominator (the pooled standard error) needs to change too if you want to see the same result. $\endgroup$ – AlexK May 25 at 1:04
  • $\begingroup$ I don't understand why the slope value and standard error of slope value are dependent. Quoting the definition, "The standard error of the regression slope, s (also called the standard error of estimate) represents the average distance that your observed values deviate from the regression line. The smaller the s value, the closer your values are to the regression line." The average spread of points from the line should be independent of slope. Unless the residuals are being measured in the y-direction rather than perpendicular to the slope? Is there a polar coord version of this test? $\endgroup$ – Adam Hoover May 26 at 23:34
  • $\begingroup$ If slope ratios are important to you, then yes you need a special test -- indeed, you almost certainly should be using a different model. But that's a rare circumstance: it doesn't justify your omnibus recommendation to avoid the standard t-test of slopes. Please note that your recommendation isn't even well-defined, because the slope depends on the units of measurement of both variables, both of which usually are arbitrary. $\endgroup$ – whuber May 30 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.