4
$\begingroup$

I'm looking for a statistical or mathematical way to test the difference between two slopes. Others have asked related questions but my problem is quite particular.

I'm running a Poisson regression of this form with X the focal predictor and Z the moderator.

Baseline:

Y = exp(b0 + b1X - b2X^2) [assume b1 and b2 are positive so that the function is concave]

Full model:

Y = exp(b0 + b1X - b2X^2 + b3ZX + b4ZX^2 + b5Z)

I am running this regression for 2 samples of a single group. My goal is to find out whether one group is significantly more sensitive to the introduction of the moderator than the other group. Any suggestions on how to test that would be awesome.

Here is what I am currently doing

After running this regression, I calculate the turning points and then the slope (derivative) of the full Poisson model at various distances 'a' from the turning point (where the slope is obviously 0). Based on this information I can estimate a simple slope line that gives me an idea of how much the introduction of the moderator Z affects the concave shape of the main effect X and X^2 on Y. I can determine this at different values for Z. The goal is to be able to explain something like "a 1 standard deviation increase of the moderator Z in sample 1, has a much stronger effect than the same increase in sample 2. The concave function between X and Y steepens thus significantly more in sample 1."

The 2 samples are not of equal size, and have different means and standard deviations for the moderator Z. I want to find out whether the estimated simple slope lines are statistically different.

Improve this question
$\endgroup$
3
  • $\begingroup$ If I understand correctly you want to see if the shape of y in function of x depends on z? Than how about a joint test H0: b3=b4=0? $\endgroup$
    – Knarpie
    Commented Nov 21, 2019 at 8:30
  • $\begingroup$ Hi @Knarpie, this is not entirely what I want. I know from the regressions that the interactions matter and that the original concave relationship between X and Y becomes steeper as Z increases. What I need to test is whether there is a statistical difference between the steepening (i.e. narrowing of the concave relationship) in the two samples. e.g. X is between 0 and 1 and the concave relationship reaches its turning point at about 0.4 (in both samples). I'm calculating the slopes of the curve at values X = 0.15, 0.2, 0.35 for Z = mean and Z = mean + 1SD. How to compare the slopes? $\endgroup$
    – SJDS
    Commented Nov 21, 2019 at 9:19
  • $\begingroup$ How about taking the derivative of Y wrt X for your model, at the different points, and test whether their difference is zero? The standard error of this test statistic can be approximated through delta method. Check this approximation in a simulation study though $\endgroup$
    – Knarpie
    Commented Nov 26, 2019 at 10:59

1 Answer 1

Reset to default
1
$\begingroup$

The proper statistical way of calculating such difference would be to add a group indicator variable interacted with the slopes that you would like to test and compare it with the model where the group indicator variable is not present. This procedure helps you to work with the full data, and the sample difference between them will be encoded in the standard errors of the group parameter's estimates.

Here is the explanation of such procedure. Suppose the first group is named group $a$ and the second one is group $b$. You should compare your full model:

$$ g(\mu) = \beta_0 + \beta_1x + \beta_2x^2 +\beta_3zx^2 + \beta_4zx + \beta_5z\tag{full model} $$

with group interactions effect:

$$ g(\mu) = \beta_0 + \beta_1x + \beta_2x^2 +\beta_3zx^2 + \beta_4zx + \beta_5z + \beta_6zx^2\cdot\mathbb{1}_{g=b} + \beta_7zx\cdot\mathbb{1}_{g=b} + \beta_8z\cdot\mathbb{1}_{g=b} + \beta_9\mathbb{1}_{g=b} $$

$\beta_9$ should be included in the model if there is a difference in the mean levels of the response variable between the two groups ($a$ or $b$). The base line of the proposed model is then in reference to group $a$. Note that if there is a difference between groups exists, the estimates for the curvature parameters are updated by the terms where $\mathbb{1}_{g=b}$ is true.

So who do you test the model? There are a number of approaches:

  • Wald test for $\beta_6$, $\beta_7$, $\beta_8$ and $\beta_9$
  • likelihood ratio tests between the full model and the proposed model
  • score tests
  • etc..
Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi Guilherme, thanks a lot for your clear explanation. I have a few additional questions. 1) Would the same procedure work if you have there are additional control variables X that exist for both samples (I presume it would not matter). 2) Can this procedure also be used when there are two moderators (Z and K) both interacting with X (I presume yes). 3) Can this work in a model with fixed effects? 4) When executing the group interaction effects as you suggests results in all insignificant betas, does this automatically mean that my hypothesis is wrong? $\endgroup$
    – SJDS
    Commented Nov 28, 2019 at 1:47
  • $\begingroup$ 5) LR test is likely to be invalid (says stata) in models with robust standard errors so I cannot use this test. $\endgroup$
    – SJDS
    Commented Nov 28, 2019 at 1:50
  • $\begingroup$ Let me address each question here. 1) yes, just have to think about how to parameterize the model. 2) You can, but since your are dealing with a lot of higher degree interactions, those types of effects are hard to be detected in a) lower sample size settings, b) high variance settings, so I would simplify the problem (such as stratifying the sample by Z or K and test within the strata for group differences) 3) Yep $\endgroup$
    – Guilherme Marthe
    Commented Nov 28, 2019 at 14:23
  • $\begingroup$ 4) No. it can mean many things. For once, you may have low sample size to detect those small differences. And remember the absence of evidence of an effect does not mean evidence of absence of the effect. 5) If in some combination of Z, K, group you don't have a lot of data, it can be numerically tricky to estimate the effects you are looking for. Some options could be to add regularization of some sort , or in a bayesian setting, include strong priors. $\endgroup$
    – Guilherme Marthe
    Commented Nov 28, 2019 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.