I have a pretty basic question for those of you familiar with finite mixture models. The simple question is: How come the coefficent estimates are so different between FMM and my OLS models. Specifically, if I partition my sample based on the FMM classification and estimate coefficents via OLS for each "class", the coefficients are quite different from the ones that are estimated directly with the finite mixture model. Does this suggest that one of the estimates must be bias? Does this result suggest something else of which I am not aware? Any help or references to investigate this issue would be greatly appreciated. Thanks
1
-
$\begingroup$ I doubt this is answerable in its current form. Can you provide example data, whatever code you used to fit these models & your output? $\endgroup$– gung - Reinstate MonicaCommented Jul 30, 2016 at 0:15
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
The FMM coefficients for class $k$ are comparable to a weighted full sample regression where the weights are the latent class posterior probabilities of class $k$ membership. The standard errors are more complicated.
Intuitively, this weighting makes sense since you don't know for sure which class a particular observation belongs to, so you weight their contribution to the expected value for that group by your estimate of the probability of belonging to that same group.
In the textbook fish sexing example, if you were sure that a fish was a dude fish, he should contribute nothing to the lady fish model. Most mixtures won't be that "separable" in practice and Cromwell's Rule will be obeyed by FMMs.
Here's an example with Stata using user-written fmm and fmmlc:
. set more off
. webuse womenwk, clear
. fmm wage educ age married, mix(normal) comp(2)
Fitting Normal regression model:
Iteration 0: log likelihood = -4181.589
Iteration 1: log likelihood = -4181.586
Iteration 2: log likelihood = -4181.586
Fitting 2 component Normal model:
Iteration 0: log likelihood = -4181.7341 (not concave)
Iteration 1: log likelihood = -4181.5898 (not concave)
Iteration 2: log likelihood = -4181.4868 (not concave)
Iteration 3: log likelihood = -4180.4487 (not concave)
Iteration 4: log likelihood = -4178.422 (not concave)
Iteration 5: log likelihood = -4177.1612 (not concave)
Iteration 6: log likelihood = -4176.2248
Iteration 7: log likelihood = -4175.1488 (not concave)
Iteration 8: log likelihood = -4174.8259
Iteration 9: log likelihood = -4174.5849
Iteration 10: log likelihood = -4174.5798
Iteration 11: log likelihood = -4174.5798
2 component Normal regression Number of obs = 1,343
Wald chi2(6) = 438.48
Log likelihood = -4174.5798 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
wage | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
component1 |
education | .9485605 .0916029 10.36 0.000 .7690222 1.128099
age | .124996 .0333556 3.75 0.000 .0596202 .1903718
married | -1.812724 1.407081 -1.29 0.198 -4.570552 .9451045
_cons | 6.278988 1.480055 4.24 0.000 3.378133 9.179843
-------------+----------------------------------------------------------------
component2 |
education | .8014959 .1383142 5.79 0.000 .5304052 1.072587
age | .2012236 .0590787 3.41 0.001 .0854314 .3170158
married | 3.499962 2.727121 1.28 0.199 -1.845097 8.845021
_cons | 5.704543 2.979757 1.91 0.056 -.1356727 11.54476
-------------+----------------------------------------------------------------
/imlogitpi1 | .869283 1.258804 0.69 0.490 -1.597928 3.336494
/lnsigma1 | 1.59161 .052347 30.40 0.000 1.489011 1.694208
/lnsigma2 | 1.626932 .0746651 21.79 0.000 1.480592 1.773273
------------------------------------------------------------------------------
sigma1 | 4.911649 .2571102 4.432711 5.442334
sigma2 | 5.088242 .3799143 4.395545 5.890103
pi1 | .7045965 .2620079 .1682714 .9656598
pi2 | .2954035 .2620079 .0343402 .8317286
------------------------------------------------------------------------------
. qui fmmlc, savec savep
. /* Compare to FMM Coefficients */
. reg wage educ age married [pw=_prob1_1]
(sum of wgt is 9.4627e+02)
Linear regression Number of obs = 1,343
F(3, 1339) = 181.49
Prob > F = 0.0000
R-squared = 0.2765
Root MSE = 4.919
------------------------------------------------------------------------------
| Robust
wage | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
education | .9485607 .0448757 21.14 0.000 .8605263 1.036595
age | .124996 .0168575 7.41 0.000 .091926 .1580659
married | -1.812725 .3192834 -5.68 0.000 -2.439075 -1.186375
_cons | 6.278987 .7822908 8.03 0.000 4.744338 7.813636
------------------------------------------------------------------------------
. reg wage educ age married [pw=_prob2_1]
(sum of wgt is 3.9673e+02)
Linear regression Number of obs = 1,343
F(3, 1339) = 192.88
Prob > F = 0.0000
R-squared = 0.3906
Root MSE = 5.0958
------------------------------------------------------------------------------
| Robust
wage | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
education | .8014961 .05736 13.97 0.000 .6889708 .9140213
age | .2012236 .0253604 7.93 0.000 .1514731 .2509741
married | 3.499962 .3773495 9.28 0.000 2.759701 4.240222
_cons | 5.70454 1.093122 5.22 0.000 3.560122 7.848958
------------------------------------------------------------------------------
. /* This is what you want to do */
. bys _class_1: reg wage educ age married
--------------------------------------------------------------------------------------------------------------------------------------------
-> _class_1 = FMM Component 1
Source | SS df MS Number of obs = 1,168
-------------+---------------------------------- F(3, 1164) = 184.82
Model | 11188.2566 3 3729.41888 Prob > F = 0.0000
Residual | 23487.9617 1,164 20.1786612 R-squared = 0.3226
-------------+---------------------------------- Adj R-squared = 0.3209
Total | 34676.2183 1,167 29.7139831 Root MSE = 4.4921
------------------------------------------------------------------------------
wage | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
education | .9643817 .0454831 21.20 0.000 .8751437 1.05362
age | .1310328 .0173743 7.54 0.000 .0969444 .1651212
married | -2.068989 .3082083 -6.71 0.000 -2.673695 -1.464283
_cons | 5.934805 .7905355 7.51 0.000 4.383772 7.485839
------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
-> _class_1 = FMM Component 2
note: married omitted because of collinearity
Source | SS df MS Number of obs = 175
-------------+---------------------------------- F(2, 172) = 131.14
Model | 2282.86052 2 1141.43026 Prob > F = 0.0000
Residual | 1497.05137 172 8.70378703 R-squared = 0.6039
-------------+---------------------------------- Adj R-squared = 0.5993
Total | 3779.91189 174 21.7236315 Root MSE = 2.9502
------------------------------------------------------------------------------
wage | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
education | 1.04502 .0716459 14.59 0.000 .9036013 1.186438
age | .1466719 .0286646 5.12 0.000 .0900921 .2032516
married | 0 (omitted)
_cons | 12.33115 1.419097 8.69 0.000 9.530066 15.13224
------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
