0
$\begingroup$

How can I test the equality of coefficients from multiple regression models?

I want to compare multiple measurement methods to see if they capture the dynamic processes of the same construct in similar ways; the hypothesis I intend to test is that the regression coefficients across models are equal because all measurement methods are equally sensitive to changes in the construct studied.

First, the linear increase of the three methods as the number of interaction increases was modelled. I used mixed effect regression models for each method with a long format data frame, as each method was assessed multiple times per participant.

However, the methods weren't assessed the same amount of time, so the data frames used for each regression model aren't of the same length. TS1 and TS2 were assessed before each interaction (12 in total), TS3 was only assessed 2 times: before the first and last interaction with the system

To then compare the coefficients, I thought about using seemingly unrelated regression (SUR) to fit the regression models together. This would allow for a comparison of the coefficients. However, I am unsure if this method is the most appropriate for the data used. To use seemingly unrelated regression, the regression models have to be based on the same data set. So the three long format datasets used for each regression model need to be combined in one data set.

Is there a more appropriate method / statistical model to test the equality of coefficient? Or is seemingly unrelated regression the best method to compare the equality of multiple regression coefficients?

If anyone has an idea, I would be truly grateful!

These are examples of the mixed effect models I used. The long-format data frames are all based on the same original dataset:

lmer(ts1 ~ interaction_num + (1|subject_id), dfts1_long)

lmer(ts2 ~ interaction_num + (1|subject_id), dfts2_long)

lmer(ts3 ~ interaction_num + (1|subject_id), dfts3_long)

$\endgroup$
8
  • $\begingroup$ Is time a real time measurement, and if so, are the three methods measured on the same absolute time scale? E.g. is, fo0r a given person, ts1 measured at t=0, t=2, t=5; ts2 measured after ts1, namely at t=6, t=10, t=11 and t=20? And ts3 after ts2? Or do they all three start at t=0? Here I assume that t=6 means "measured at day 6" or in some other time unit. I was thinking about a extended lmer model, using data from all three methods in one single model, but need more info about the time variable. $\endgroup$
    – BenP
    Commented Dec 7, 2023 at 16:37
  • $\begingroup$ So, for a given person, ts1 is measured a t=0, t=1, t=2, t=3, t=4 ... t=11. The same applied for ts2 as they were measured at the same time. However, for ts3 the measurement of "time" were t=0 and t=11. So 12 measures for ts1 and ts2, but only 2 for ts3. The variable "time" is generated by the transformation of the dataset from wide to long. In the original dataset, the variable names were ts1_0 for the measure of ts1 at t=0, ts1_1 for t=1 ... If that helps clarify the structure $\endgroup$
    – kleinemaus
    Commented Dec 7, 2023 at 19:59
  • $\begingroup$ So, you measured ts1 etc. at real time points. What was the unit in which time was measured? Hours, days, months? And you want to model a linear increase (or decrease) in ts1 as time proceeds from, say, day 0 to 11? Do I interpret this correct? $\endgroup$
    – BenP
    Commented Dec 7, 2023 at 22:39
  • $\begingroup$ The time variable describes before which interaction with a system the construct was measured. So ts1 at t=0 is the assessment before the first interaction; t=1 is before the second interaction, ... It is not a real measure of time in a unit like seconds or hours. Sorry for the confusion. And yes exactly, I would like to model the linear increase in ts1 /ts2/ts3 as time (number of interactions) increases. $\endgroup$
    – kleinemaus
    Commented Dec 8, 2023 at 7:55
  • $\begingroup$ I do not believe this question to be about programming, debugging. Also, it certainly is not about a routine operation to be performed. It deals with finding the right statistical model for relatively complex data. With a multilevel model using e.g. lmer in R, it must be possible to formulate a suitable model. I would li8ke to help the OP with doing this. So please could you open the question again? $\endgroup$
    – BenP
    Commented Dec 8, 2023 at 10:34

1 Answer 1

0
$\begingroup$

You could combine the three datasets, to make one (very) long dataset. For a given subject_id you could then have e.g. 26 records or observations, 12 for method 1, 12 for method 2, and 2 for method 3. You should make a factor variable, "method", valued 1, 2 or 3, depending on the method to which each record (row) in your data frame belongs. So for this subject the factor "method" would be 12 times 1, 12 times 2, and 2 times 3. Your dependent variable is "ts" and contains the 26 values for the 3 methods. Finally, variable "interaction_num" takes values 1 to 12. Instead of the three models you proposed in your post, you could use one single model now:

model1 <- lmer(ts ~ method*interaction_num + (0+method||subject_id)

This would give you (almost) the same results as running the separate models you proposed. In the fixed effects results you will see the regression coefficient for interaction_num, which holds for method 1 (the reference of the factor method). The term method2:interaction_num shows you how much for method 2 the effect of interaction_num deviates from the effect for method 1. This is probably what you are most interested in. Same for method3:interaction_num. The effect of method2, shows you the ts difference between method2 vs. method1 for interaction_num being 0. This is not meaningful, but if you would code interaction_num from 0-11 instead of 1-12 then it would have a meaningful interpretation: difference between method 2 and 1 for the first interaction.

There will be three different random intercept variances, one for each method. The "||" in (0+method||subject_id) means that these three random intercepts are uncorrelated. This is only done above to let the results be more similar to the ones you would have if you would run your separate models. However, you probably will get a better model fit (deviance) if you would use (0+method|subject_id) so that correlations between the three random intercepts are allowed and estimated.

This model1 will estimate only one single residual variance! This is different from the three models you proposed, because these will give you three different residual variances. So, model1 comes with the assumption that the within-subject variance for each method is the same. And hence, the results of model1 will almost (but not exactly) be the same as those of your separate models, if indeed these residual variances are close. You can (afaik) with lme from package nlme specify that you also want to have three different residual variances.

The model1 assumes, for a given method, the same total variance in your data at each interaction number and also the same covariance (or correlation) between a pair of interaction numbers. This is called a compound symmetry covariance structure. This may not be plausible for your data. If you would let the interaction_number have a random effect for each person/method combination, you would have a more flexible covariance structure. That would be possible for method1 and method2, but not for method3, because you only have two observations there.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.