1
$\begingroup$

I have a dataset of census tracts (say 4000 tracts). There are 2500 tract with high density. Now I can run regression on the whole dataset. Or I can also run regression on dense part or low dense part SEPARATELY. Now how can I figure out if this splitting (only high density or only low density) is yielding better models, or the previous model (with the whole dataset) is better?

can I compare this with the likelihood ratio test?

(same target variable, the same set of independent variable; nothing with train test things.)

$\endgroup$
3
  • $\begingroup$ if you fit them separately what would you want to compare? I am don't quite understand what kind of conclusions a likelihood ratio test will get you $\endgroup$
    – StupidWolf
    Apr 13, 2020 at 10:30
  • $\begingroup$ say I am interested to see the impact of A on B. Now the parameter for 4000 tracts came 0.34 while the parameter for those dense 2500 tracts came 1.25. now two things can happen,(1) the first one is under fitted (more generalized) and the second one is good, OR (2) the first one is good and the second one is overfitted (as giving an estimate for more specified sample sizes). How can I make comment on this division? $\endgroup$ Apr 13, 2020 at 22:32
  • $\begingroup$ Ok I can write an answer below, since it might be too long for a discussion, we can take it from there $\endgroup$
    – StupidWolf
    Apr 13, 2020 at 22:42

1 Answer 1

0
$\begingroup$

If i get you correct, you fit a regression and get a coefficient (which you refer to as parameter). You have done it for all observations and separately for two groups, and your question is whether you can compare whether fitting separately or together as one group, is a better model so as to say.

If you have a linear model already, let's say your dependent variable is y, independent is x, and the group is a categorical variable called G, the formula will be (below is what's normally used in R or python):

y ~ x + G + x:G

You can do an anova or t.test for the interaction term x:G, this will tell you whether the coefficient significantly differs between the two groups. I think this is more straight-forward.

You cannot really comment on whether one is overfitting or underfitting, because there is no prediction etc involved. You fit everything to the model. If the interaction is significant, you can say there is evidence to suggest the parameter x, is different between the two groups in G.

The "comparing likelihood" you are referring to is possible, so if would be a LRT test, something like:

m1: 1 paramter
m2: 2 paramter

LR = -2*(logLik(m2) - logLik(m1))

And you test it with a chi square, 1 degree of freedom.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.