1
$\begingroup$

I have a crime data of 300 local divisions (level 1 units) unevenly distributed across 40 districts (level 2 units). I call it Data1. I decided to use cluster analysis to reclassify the level 1 units to obtain 40 new clusters of similar crime concentrations, and so new cluster membership were obtained different from Data1. I call the second classification as Data2. Note that the same level 1 units were maintained in the two data sets, but with differences in cluster membership. I used LMM to model the two data sets.

  1. If the conditional AIC (cAIC) for Data1 (model1) is 1032.4 and the cAIC for Data2 (model2) is 872.1, can I say model2 better fit the data than model1?

  2. I reclassify the initial data set and got 30 new optimal clusters and name it Data3.If the cAIC is 903.7 from model3, can I say model2 is better than model3?

Kindly, help me with some literatures on the topic.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

If I've understood correctly, I don't think you can compare AIC between these models as they are not run all on the same dataset. In the linear mixed models, the grouping variable is different for each model.

If you really want to compare the fit you might look at measures of predictive accuracy such as root mean sequared error. However, the procedure you have outlined will likely lead to overfitting. You could conceivably choose all possible numbers of clusters, fit a model on each one, and find the model with the "best" fit. In such a situation I would expect that model to generalise very poorly to new data.

I would recommend sticking with the existing groupings, which reflect reality, rather than a data driven approach.

Improve this answer
$\endgroup$
1
  • $\begingroup$ @Yusuf Does this answer your question ? If so then please consider marking it as a the accepted answer and (if you haven't already) upvoting it. If not, please could you let us know why so that it can be improved $\endgroup$
    – Robert Long
    Commented Jun 4, 2021 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.