1
$\begingroup$

I have a pretty basic question for those of you familiar with finite mixture models. The simple question is: How come the coefficent estimates are so different between FMM and my OLS models. Specifically, if I partition my sample based on the FMM classification and estimate coefficents via OLS for each "class", the coefficients are quite different from the ones that are estimated directly with the finite mixture model. Does this suggest that one of the estimates must be bias? Does this result suggest something else of which I am not aware? Any help or references to investigate this issue would be greatly appreciated. Thanks

$\endgroup$
  • $\begingroup$ I doubt this is answerable in its current form. Can you provide example data, whatever code you used to fit these models & your output? $\endgroup$ – gung Jul 30 '16 at 0:15
1
$\begingroup$

The FMM coefficients for class $k$ are comparable to a weighted full sample regression where the weights are the latent class posterior probabilities of class $k$ membership. The standard errors are more complicated.

Intuitively, this weighting makes sense since you don't know for sure which class a particular observation belongs to, so you weight their contribution to the expected value for that group by your estimate of the probability of belonging to that same group.

In the textbook fish sexing example, if you were sure that a fish was a dude fish, he should contribute nothing to the lady fish model. Most mixtures won't be that "separable" in practice and Cromwell's Rule will be obeyed by FMMs.

Here's an example with Stata using user-written fmm and fmmlc:

. set more off        

. webuse womenwk, clear        

. fmm wage educ age married, mix(normal) comp(2)

Fitting Normal regression model:

Iteration 0:   log likelihood =  -4181.589  
Iteration 1:   log likelihood =  -4181.586  
Iteration 2:   log likelihood =  -4181.586  

Fitting 2 component Normal model:

Iteration 0:   log likelihood = -4181.7341  (not concave)
Iteration 1:   log likelihood = -4181.5898  (not concave)
Iteration 2:   log likelihood = -4181.4868  (not concave)
Iteration 3:   log likelihood = -4180.4487  (not concave)
Iteration 4:   log likelihood =  -4178.422  (not concave)
Iteration 5:   log likelihood = -4177.1612  (not concave)
Iteration 6:   log likelihood = -4176.2248  
Iteration 7:   log likelihood = -4175.1488  (not concave)
Iteration 8:   log likelihood = -4174.8259  
Iteration 9:   log likelihood = -4174.5849  
Iteration 10:  log likelihood = -4174.5798  
Iteration 11:  log likelihood = -4174.5798  

2 component Normal regression                   Number of obs     =      1,343
                                                Wald chi2(6)      =     438.48
Log likelihood = -4174.5798                     Prob > chi2       =     0.0000

------------------------------------------------------------------------------
        wage |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
component1   |
   education |   .9485605   .0916029    10.36   0.000     .7690222    1.128099
         age |    .124996   .0333556     3.75   0.000     .0596202    .1903718
     married |  -1.812724   1.407081    -1.29   0.198    -4.570552    .9451045
       _cons |   6.278988   1.480055     4.24   0.000     3.378133    9.179843
-------------+----------------------------------------------------------------
component2   |
   education |   .8014959   .1383142     5.79   0.000     .5304052    1.072587
         age |   .2012236   .0590787     3.41   0.001     .0854314    .3170158
     married |   3.499962   2.727121     1.28   0.199    -1.845097    8.845021
       _cons |   5.704543   2.979757     1.91   0.056    -.1356727    11.54476
-------------+----------------------------------------------------------------
 /imlogitpi1 |    .869283   1.258804     0.69   0.490    -1.597928    3.336494
   /lnsigma1 |    1.59161    .052347    30.40   0.000     1.489011    1.694208
   /lnsigma2 |   1.626932   .0746651    21.79   0.000     1.480592    1.773273
------------------------------------------------------------------------------
      sigma1 |   4.911649   .2571102                      4.432711    5.442334
      sigma2 |   5.088242   .3799143                      4.395545    5.890103
         pi1 |   .7045965   .2620079                      .1682714    .9656598
         pi2 |   .2954035   .2620079                      .0343402    .8317286
------------------------------------------------------------------------------

. qui fmmlc, savec savep     

. /* Compare to FMM Coefficients */
. reg wage educ age married [pw=_prob1_1]
(sum of wgt is   9.4627e+02)

Linear regression                               Number of obs     =      1,343
                                                F(3, 1339)        =     181.49
                                                Prob > F          =     0.0000
                                                R-squared         =     0.2765
                                                Root MSE          =      4.919

------------------------------------------------------------------------------
             |               Robust
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   education |   .9485607   .0448757    21.14   0.000     .8605263    1.036595
         age |    .124996   .0168575     7.41   0.000      .091926    .1580659
     married |  -1.812725   .3192834    -5.68   0.000    -2.439075   -1.186375
       _cons |   6.278987   .7822908     8.03   0.000     4.744338    7.813636
------------------------------------------------------------------------------

. reg wage educ age married [pw=_prob2_1]      
(sum of wgt is   3.9673e+02)

Linear regression                               Number of obs     =      1,343
                                                F(3, 1339)        =     192.88
                                                Prob > F          =     0.0000
                                                R-squared         =     0.3906
                                                Root MSE          =     5.0958

------------------------------------------------------------------------------
             |               Robust
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   education |   .8014961     .05736    13.97   0.000     .6889708    .9140213
         age |   .2012236   .0253604     7.93   0.000     .1514731    .2509741
     married |   3.499962   .3773495     9.28   0.000     2.759701    4.240222
       _cons |    5.70454   1.093122     5.22   0.000     3.560122    7.848958
------------------------------------------------------------------------------

. /* This is what you want to do */
. bys _class_1: reg wage educ age married

--------------------------------------------------------------------------------------------------------------------------------------------
-> _class_1 = FMM Component 1

      Source |       SS           df       MS      Number of obs   =     1,168
-------------+----------------------------------   F(3, 1164)      =    184.82
       Model |  11188.2566         3  3729.41888   Prob > F        =    0.0000
    Residual |  23487.9617     1,164  20.1786612   R-squared       =    0.3226
-------------+----------------------------------   Adj R-squared   =    0.3209
       Total |  34676.2183     1,167  29.7139831   Root MSE        =    4.4921

------------------------------------------------------------------------------
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   education |   .9643817   .0454831    21.20   0.000     .8751437     1.05362
         age |   .1310328   .0173743     7.54   0.000     .0969444    .1651212
     married |  -2.068989   .3082083    -6.71   0.000    -2.673695   -1.464283
       _cons |   5.934805   .7905355     7.51   0.000     4.383772    7.485839
------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------------------
-> _class_1 = FMM Component 2
note: married omitted because of collinearity

      Source |       SS           df       MS      Number of obs   =       175
-------------+----------------------------------   F(2, 172)       =    131.14
       Model |  2282.86052         2  1141.43026   Prob > F        =    0.0000
    Residual |  1497.05137       172  8.70378703   R-squared       =    0.6039
-------------+----------------------------------   Adj R-squared   =    0.5993
       Total |  3779.91189       174  21.7236315   Root MSE        =    2.9502

------------------------------------------------------------------------------
        wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   education |    1.04502   .0716459    14.59   0.000     .9036013    1.186438
         age |   .1466719   .0286646     5.12   0.000     .0900921    .2032516
     married |          0  (omitted)
       _cons |   12.33115   1.419097     8.69   0.000     9.530066    15.13224
------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------------------
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.