# finite mixture models versus segmented OLS

I have a pretty basic question for those of you familiar with finite mixture models. The simple question is: How come the coefficent estimates are so different between FMM and my OLS models. Specifically, if I partition my sample based on the FMM classification and estimate coefficents via OLS for each "class", the coefficients are quite different from the ones that are estimated directly with the finite mixture model. Does this suggest that one of the estimates must be bias? Does this result suggest something else of which I am not aware? Any help or references to investigate this issue would be greatly appreciated. Thanks

• I doubt this is answerable in its current form. Can you provide example data, whatever code you used to fit these models & your output? – gung Jul 30 '16 at 0:15

The FMM coefficients for class $$k$$ are comparable to a weighted full sample regression where the weights are the latent class posterior probabilities of class $$k$$ membership. The standard errors are more complicated.

Intuitively, this weighting makes sense since you don't know for sure which class a particular observation belongs to, so you weight their contribution to the expected value for that group by your estimate of the probability of belonging to that same group.

In the textbook fish sexing example, if you were sure that a fish was a dude fish, he should contribute nothing to the lady fish model. Most mixtures won't be that "separable" in practice and Cromwell's Rule will be obeyed by FMMs.

Here's an example with Stata using user-written fmm and fmmlc:

. set more off

. webuse womenwk, clear

. fmm wage educ age married, mix(normal) comp(2)

Fitting Normal regression model:

Iteration 0:   log likelihood =  -4181.589
Iteration 1:   log likelihood =  -4181.586
Iteration 2:   log likelihood =  -4181.586

Fitting 2 component Normal model:

Iteration 0:   log likelihood = -4181.7341  (not concave)
Iteration 1:   log likelihood = -4181.5898  (not concave)
Iteration 2:   log likelihood = -4181.4868  (not concave)
Iteration 3:   log likelihood = -4180.4487  (not concave)
Iteration 4:   log likelihood =  -4178.422  (not concave)
Iteration 5:   log likelihood = -4177.1612  (not concave)
Iteration 6:   log likelihood = -4176.2248
Iteration 7:   log likelihood = -4175.1488  (not concave)
Iteration 8:   log likelihood = -4174.8259
Iteration 9:   log likelihood = -4174.5849
Iteration 10:  log likelihood = -4174.5798
Iteration 11:  log likelihood = -4174.5798

2 component Normal regression                   Number of obs     =      1,343
Wald chi2(6)      =     438.48
Log likelihood = -4174.5798                     Prob > chi2       =     0.0000

------------------------------------------------------------------------------
wage |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
component1   |
education |   .9485605   .0916029    10.36   0.000     .7690222    1.128099
age |    .124996   .0333556     3.75   0.000     .0596202    .1903718
married |  -1.812724   1.407081    -1.29   0.198    -4.570552    .9451045
_cons |   6.278988   1.480055     4.24   0.000     3.378133    9.179843
-------------+----------------------------------------------------------------
component2   |
education |   .8014959   .1383142     5.79   0.000     .5304052    1.072587
age |   .2012236   .0590787     3.41   0.001     .0854314    .3170158
married |   3.499962   2.727121     1.28   0.199    -1.845097    8.845021
_cons |   5.704543   2.979757     1.91   0.056    -.1356727    11.54476
-------------+----------------------------------------------------------------
/imlogitpi1 |    .869283   1.258804     0.69   0.490    -1.597928    3.336494
/lnsigma1 |    1.59161    .052347    30.40   0.000     1.489011    1.694208
/lnsigma2 |   1.626932   .0746651    21.79   0.000     1.480592    1.773273
------------------------------------------------------------------------------
sigma1 |   4.911649   .2571102                      4.432711    5.442334
sigma2 |   5.088242   .3799143                      4.395545    5.890103
pi1 |   .7045965   .2620079                      .1682714    .9656598
pi2 |   .2954035   .2620079                      .0343402    .8317286
------------------------------------------------------------------------------

. qui fmmlc, savec savep

. /* Compare to FMM Coefficients */
. reg wage educ age married [pw=_prob1_1]
(sum of wgt is   9.4627e+02)

Linear regression                               Number of obs     =      1,343
F(3, 1339)        =     181.49
Prob > F          =     0.0000
R-squared         =     0.2765
Root MSE          =      4.919

------------------------------------------------------------------------------
|               Robust
wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
education |   .9485607   .0448757    21.14   0.000     .8605263    1.036595
age |    .124996   .0168575     7.41   0.000      .091926    .1580659
married |  -1.812725   .3192834    -5.68   0.000    -2.439075   -1.186375
_cons |   6.278987   .7822908     8.03   0.000     4.744338    7.813636
------------------------------------------------------------------------------

. reg wage educ age married [pw=_prob2_1]
(sum of wgt is   3.9673e+02)

Linear regression                               Number of obs     =      1,343
F(3, 1339)        =     192.88
Prob > F          =     0.0000
R-squared         =     0.3906
Root MSE          =     5.0958

------------------------------------------------------------------------------
|               Robust
wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
education |   .8014961     .05736    13.97   0.000     .6889708    .9140213
age |   .2012236   .0253604     7.93   0.000     .1514731    .2509741
married |   3.499962   .3773495     9.28   0.000     2.759701    4.240222
_cons |    5.70454   1.093122     5.22   0.000     3.560122    7.848958
------------------------------------------------------------------------------

. /* This is what you want to do */
. bys _class_1: reg wage educ age married

--------------------------------------------------------------------------------------------------------------------------------------------
-> _class_1 = FMM Component 1

Source |       SS           df       MS      Number of obs   =     1,168
-------------+----------------------------------   F(3, 1164)      =    184.82
Model |  11188.2566         3  3729.41888   Prob > F        =    0.0000
Residual |  23487.9617     1,164  20.1786612   R-squared       =    0.3226
Total |  34676.2183     1,167  29.7139831   Root MSE        =    4.4921

------------------------------------------------------------------------------
wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
education |   .9643817   .0454831    21.20   0.000     .8751437     1.05362
age |   .1310328   .0173743     7.54   0.000     .0969444    .1651212
married |  -2.068989   .3082083    -6.71   0.000    -2.673695   -1.464283
_cons |   5.934805   .7905355     7.51   0.000     4.383772    7.485839
------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------------------
-> _class_1 = FMM Component 2
note: married omitted because of collinearity

Source |       SS           df       MS      Number of obs   =       175
-------------+----------------------------------   F(2, 172)       =    131.14
Model |  2282.86052         2  1141.43026   Prob > F        =    0.0000
Residual |  1497.05137       172  8.70378703   R-squared       =    0.6039