Timeline for How to prove that a separate linear model for each class is equivalent to using interaction with the class?
Current License: CC BY-SA 3.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 28, 2017 at 13:47 | comment | added | whuber♦ | @Amoeba You are correct: in the case of least squares regression, the objective function is the same in both cases and so the coefficient estimates must be the same. | |
| May 28, 2017 at 10:38 | comment | added | amoeba | @whuber I understand that the error variance will be different and so everything dependent on it (like t-tests for individual parameters) can differ too. But how can coefficient estimates differ? In the example quoted by hxd1011 in the beginning of this Q, we are talking about $y=ax+b$ for each of the $i=1\ldots k$ groups separately vs. $y=a_i x + b_i$ for all groups together; shouldn't this produce exactly equivalent coefficients? | |
| May 26, 2017 at 17:41 | comment | added | Haitao Du | thanks for @whuber's comment in the chat: Separate linear regressions will result in different estimates of the variance of the error terms. A single regression with an interaction will produce a single ("pooled") estimate of the error terms. In so doing, it's likely to produce slightly different coefficient estimates, too. Thus, your choice of how to proceed should be determined in large part by your assumptions about homoscedasticity and any evidence to the contrary. It's really just a generalization of the distinction between t-tests assuming equal or unequal variances. | |
| May 26, 2017 at 17:00 | comment | added | whuber♦ | This question is answered at stats.stackexchange.com/questions/13112. My answer there demonstrates mathematically that the t-tests to compare the main effects differ between the models. At the end of an answer to a related question I also discuss this issue briefly: stats.stackexchange.com/a/12809/919. No algebra is needed at all: when you write down the two models, you see that the two-regression model has an extra (identifiable) parameter, which ought to make everything clear. | |
| May 3, 2017 at 14:11 | history | edited | amoeba | CC BY-SA 3.0 |
added 17 characters in body; edited tags; edited title
|
| May 1, 2017 at 2:05 | comment | added | GeoMatt22 | OK. I was not sure if the "automagic" re-coding R does under the hood was obscuring this. (I don't have much R experience, so not sure what can be queried.) | |
| May 1, 2017 at 1:53 | comment | added | Haitao Du | @GeoMatt22 I tried to see the design matrix and dummy coding, I can see why they are the same, but just wondering if there is there any elegant math proof with matrix algebra. | |
| Apr 30, 2017 at 17:24 | answer | added | Björn | timeline score: 4 | |
| Apr 30, 2017 at 3:15 | comment | added | GeoMatt22 | I am not familiar with R, but can LM give you the design matrix? I would think you could look at it's sparsity structure (i.e. non-zeros) to investigate. (Dummy coding is like one-hot, right? So no zeros will come out of dot-products due to +/- cancellations, so you can ignore sign?) | |
| Apr 29, 2017 at 18:25 | history | asked | Haitao Du | CC BY-SA 3.0 |