0
$\begingroup$

I am trying to implement a Linear Mixed-Effects Model in Matlab. I have many repeated measures of some features in a longitudinal data set of 51 people. I considered a random intercept that varies by subject, and all the other features as fixed-effects.

These are the results that I got:

Linear mixed-effects model fit by REML

Model information:
    Number of observations            1432
    Fixed effects coefficients          41
    Random effects coefficients         51
    Covariance parameters                2

Formula:
    Linear Mixed Formula with 41 predictors.

Model fit statistics:
    AIC       BIC       LogLikelihood    Deviance
    -37617    -37392    18852            -37703  

Fixed effects coefficients (95% CIs):
    Name                                       Estimate       SE            tStat          DF      pValue        Lower          Upper     
    '(Intercept)'                                 0.058824       0.00432         13.617    1391    9.9401e-40        0.05035      0.067299
    'walknums'                                  3.7027e-15    1.7968e-09     2.0607e-06    1391             1    -3.5248e-09    3.5248e-09
    'spd_MED'                                   1.3314e-14    1.5799e-09     8.4271e-06    1391       0.99999    -3.0992e-09    3.0993e-09
    'spd_IQR'                                  -5.2216e-15    7.2452e-10     -7.207e-06    1391       0.99999    -1.4213e-09    1.4213e-09
    'TimesOut'                                  6.9186e-17    1.2472e-08     5.5474e-09    1391             1    -2.4466e-08    2.4466e-08
    'timeout_mins'                             -5.3738e-16     1.528e-10    -3.5169e-06    1391             1    -2.9974e-10    2.9974e-10
    'cpu_sessions'                               -5.37e-14     1.858e-08    -2.8902e-06    1391             1    -3.6448e-08    3.6448e-08
    'cpu_time'                                 -1.3575e-15    6.3484e-10    -2.1383e-06    1391             1    -1.2453e-09    1.2453e-09
    'TransNum'                                  6.4053e-16    1.4996e-10     4.2712e-06    1391             1    -2.9418e-10    2.9418e-10
    'SensorNum'                                 2.0057e-16    6.1782e-11     3.2465e-06    1391             1     -1.212e-10     1.212e-10
    'TST__Total_Sleep_Time_'                    1.7449e-17    2.9183e-10     5.9794e-08    1391             1    -5.7247e-10    5.7247e-10
    'TTIB__Total_Time_in_Bed_'                  2.2715e-16    1.9261e-10     1.1793e-06    1391             1    -3.7785e-10    3.7785e-10
    'LATENCY'                                  -8.2113e-16     7.567e-10    -1.0851e-06    1391             1    -1.4844e-09    1.4844e-09
    'WASO__Wake_After_Sleep_Onset_'             3.2435e-16    1.2723e-10     2.5494e-06    1391             1    -2.4958e-10    2.4958e-10
    'MIB__Motion_In_Bed_'                       2.2456e-16    8.3319e-10     2.6952e-07    1391             1    -1.6344e-09    1.6344e-09
    'UP_TIMES__Times_up_at_night_'              1.7083e-13     2.053e-08     8.3209e-06    1391       0.99999    -4.0273e-08    4.0273e-08
    'BATHROOM_VISITS'                          -2.1346e-13    2.8767e-08    -7.4202e-06    1391       0.99999    -5.6432e-08    5.6431e-08
    'TIME_TO_BED'                               7.1692e-20    5.5363e-15     1.2949e-05    1391       0.99999     -1.086e-14    1.0861e-14
    'INITIAL_LATENCY'                          -1.1546e-15    6.1742e-10      -1.87e-06    1391             1    -1.2112e-09    1.2112e-09
    'time_walking'                             -9.5444e-17    2.3914e-11    -3.9911e-06    1391             1    -4.6912e-11    4.6912e-11
    'timeout_MED'                              -4.0725e-16    1.4192e-10    -2.8695e-06    1391             1     -2.784e-10     2.784e-10
    'cpu_time_MED'                             -1.2468e-15    6.3405e-10    -1.9664e-06    1391             1    -1.2438e-09    1.2438e-09
    'TTIB_TST_ratio'                           -4.8356e-13     1.823e-07    -2.6526e-06    1391             1    -3.5761e-07    3.5761e-07
    'WASO_TST_ratio'                           -3.2172e-14    1.6291e-08    -1.9748e-06    1391             1    -3.1958e-08    3.1958e-08
    'UP_TIMES_TST_ratio'                        1.2565e-12     4.374e-06     2.8727e-07    1391             1    -8.5804e-06    8.5804e-06
    'BATHROOM_UP_TIMES_ratio'                   1.7295e-13    3.2722e-08     5.2856e-06    1391             1    -6.4189e-08    6.4189e-08
    'HOF_walknums'                             -3.6037e-18    2.0277e-11    -1.7772e-07    1391             1    -3.9776e-11    3.9776e-11
    'HOF_spd_MED'                              -9.2077e-17     1.138e-11    -8.0912e-06    1391       0.99999    -2.2324e-11    2.2323e-11
    'HOF_spd_IQR'                               3.1001e-17    6.3941e-12     4.8484e-06    1391             1    -1.2543e-11    1.2543e-11
    'HOF_TimesOut'                              2.8602e-17    1.1486e-09     2.4903e-08    1391             1    -2.2531e-09    2.2531e-09
    'HOF_timeout_mins'                          6.6336e-19    1.8415e-13     3.6022e-06    1391             1    -3.6125e-13    3.6125e-13
    'HOF_cpu_sessions'                          3.3945e-15      1.43e-09     2.3738e-06    1391             1    -2.8052e-09    2.8052e-09
    'HOF_cpu_time'                              3.2927e-18    1.4125e-12     2.3311e-06    1391             1    -2.7709e-12    2.7709e-12
    'HOF_TransNum'                             -1.6505e-19    1.4104e-13    -1.1703e-06    1391             1    -2.7668e-13    2.7668e-13
    'HOF_SensorNum'                            -8.3677e-20     1.743e-14    -4.8007e-06    1391             1    -3.4192e-14    3.4192e-14
    'HOF_TST__Total_Sleep_Time_'               -1.6576e-19    1.1649e-13    -1.4229e-06    1391             1    -2.2851e-13    2.2851e-13
    'HOF_LATENCY'                               1.0214e-17    7.1938e-12     1.4198e-06    1391             1    -1.4112e-11    1.4112e-11
    'HOF_WASO__Wake_After_Sleep_Onset_'        -4.0384e-19    2.3504e-13    -1.7182e-06    1391             1    -4.6108e-13    4.6108e-13
    'HOF_MIB__Motion_In_Bed_'                  -5.9631e-18    6.7789e-12    -8.7966e-07    1391             1    -1.3298e-11    1.3298e-11
    'HOF_UP_TIMES__Times_up_at_night_'         -7.9758e-15    1.8101e-09    -4.4062e-06    1391             1    -3.5508e-09    3.5508e-09
    'HOF_BATHROOM_VISITS'                       2.6271e-14     5.384e-09     4.8795e-06    1391             1    -1.0562e-08    1.0562e-08

Random effects covariance parameters (95% CIs): Group: subject_ID_left (51 Levels)
    Name1                Name2                Type         Estimate    Lower       Upper   
    '(Intercept)'        '(Intercept)'        'std'        0.030846    0.029737    0.031997

Group: Error
    Name             Estimate     Lower         Upper     
    'Res Std'        1.543e-07    1.2435e-07    1.9147e-07
  • Does this mean that none of my features is significant for the prediction of the result vector?
  • Should I take all of them out of my model, or should I consider other combinations?
  • Should I consider any other random effect?

Thank you for your help!

$\endgroup$
  • $\begingroup$ Something is very wrong. Every p value is exactly 1 except the intercept. You need to diagnose the problem I'd look at some graphs and check the code you used with a MATLAB expert and so on. $\endgroup$ – Peter Flom May 4 '16 at 10:59
  • 1
    $\begingroup$ @PeterFlom: Mabye due to extreme multicollinearity or just in general very poor feature construction (ridiculous scales etc) or some problem with degrees of freedom. $\endgroup$ – Michael M May 4 '16 at 11:45
  • $\begingroup$ It's hard to tell what is going wrong, but something is. $\endgroup$ – Peter Flom May 4 '16 at 11:50
2
$\begingroup$

You ask three questions following your model output.

Regarding the first question, the fact that the non-intercept covariates are not significant does not mean that they are useless for prediction. If your dataset is not very large (you have 51 people and 41 predictors), then fitting a model this large and isolating the effects of the different covariates will be difficult. If some are not informative, they can obscure the importance of others, just like in fixed effects models.

Regarding the second question, you should try other combinations. Try to focus on the variables that you think should be the most informative first. Firstly, do you expect them to come in as a main effect, and secondly, could you expect some kind of interaction between them?

You might consider another random effect if there is another grouping that the individuals can be placed into, like they belong to a set of 15 different schools, for example. Otherwise, I would first focus on trying to improve the structure of your fixed effects model.

$\endgroup$
  • $\begingroup$ Ordinarily, I'd agree with you but look at the p values - they are all .999 or 1. Something more fundamental is wrong here. $\endgroup$ – Peter Flom May 4 '16 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.