6
$\begingroup$

For my thesis there's a big chance that I will need some sort of mixed-effects specification. I have some (non-syntax) experience with SPSS but feel that it won't suffice for my analysis. I have very basic knowledge in Stata and decided to experiment more with that package.

I decided to try and replicate results from SPSS in Stata for a basic model. I have data on 4059 students in 65 schools, investigating the influence of entry level score (standlrt) of students on their final exam score (normexam).

In a previously followed course which had a brief introduction to multilevel modeling, my teacher provided me with a syntax in SPSS.

MIXED
  normexam  WITH standlrt
  /FIXED = standlrt 
  /PRINT = SOLUTION TESTCOV
 /RANDOM INTERCEPT standlrt  | SUBJECT(school) COVTYPE(VC) .

Now I tried replicating these results in Stata but the results are not consistent. Magnitude and sometimes even sign of the betas differ.

First I use xtset school to indicate that my data is clustered. Then I use

xtmixed normexam standlrt || school: standlrt .

What may be the cause of these inconsistent results?

Thanks in advance!

ps. this is not homework, and I hope I specified my first question properly.

pps. a possibility may be that the 'problem' has multiple optima but I don't think this is the case in such a basic model, also because it's an uni-variate regression. Also, the iterative procedures performed while estimating may have different results, but I only think this would have big effects like sign changes.

EDIT

This is my Stata output

xtmixed normexam standlrt || school: standlrt

Mixed-effects REML regression                   Number of obs      =      4059
Group variable: school                          Number of groups   =        65

                                                Obs per group: min =         2
                                                               avg =      62.4
                                                               max =       198


                                                Wald chi2(1)       =    768.21
Log restricted-likelihood = -4667.8385          Prob > chi2        =    0.0000

------------------------------------------------------------------------------
    normexam |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    standlrt |   .5570213   .0200971    27.72   0.000     .5176317    .5964108
       _cons |  -.0080944   .0400842    -0.20   0.840     -.086658    .0704691
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
school: Independent          |
                sd(standlrt) |   .1214197   .0191066      .0891958    .1652852
                   sd(_cons) |   .3032317   .0309434      .2482638    .3703699
-----------------------------+------------------------------------------------
                sd(Residual) |   .7440605   .0083943      .7277885    .7606962
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(2) =   438.60   Prob > chi2 = 0.0000

And this is my SPSS output

-2 Restricted Log Likelihood    9335,677

Type III Tests of Fixed Effects(a)
|---------|------------|--------------|-------|----|
|Source   |Numerator df|Denominator df|F      |Sig.|
|---------|------------|--------------|-------|----|
|Intercept|1           |60,466        |,041   |,841|
|---------|------------|--------------|-------|----|
|standlrt |1           |56,936        |768,207|,000|
|---------|------------|--------------|-------|----|
a. Dependent Variable: normexam = final exam scores.


Estimates of Fixed Effects(a)
|---------|--------|----------|------|------|----|-----------------------------------|
|Parameter|Estimate|Std. Error|df    |t     |Sig.|95% Confidence Interval            |
|         |        |          |      |      |    |-----------------------|-----------|
|         |        |          |      |      |    |Lower Bound            |Upper Bound|
|---------|--------|----------|------|------|----|-----------------------|-----------|
|Intercept|-,008094|,040084   |60,466|-,202 |,841|-,088262               |,072073    |
|---------|--------|----------|------|------|----|-----------------------|-----------|
|standlrt |,557021 |,020097   |56,936|27,717|,000|,516777                |,597266    |
|---------|--------|----------|------|------|----|-----------------------|-----------|
a. Dependent Variable: normexam = final exam scores.


Covariance Parameters

Estimates of Covariance Parameters(a)
|-------------------------------------|--------|----------|------|----|-----------------------------------|
|Parameter                            |Estimate|Std. Error|Wald Z|Sig.|95% Confidence Interval            |
|                            |--------|        |          |      |    |-----------------------|-----------|
|                                     |        |          |      |    |Lower Bound            |Upper Bound|
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|Residual                             |,553626 |,012492   |44,319|,000|,529676                |,578659    |
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|Intercept [subject = school]|Variance|,091949 |,018766   |4,900 |,000|,061635                |,137174    |
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|standlrt [subject = school] |Variance|,014743 |,004640   |3,177 |,001|,007956                |,027319    |
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
a. Dependent Variable: normexam = final exam scores.

As you can see, the log likelihoods are the same. Additionally, the fixed effects tables are the same. However the random effects are different. I'm not very skilled in interpretation yet but the results seem to differ.

These are the settings for the variance-covariance matrix

 Model
      covariance(vartype)    variance-covariance structure of the random
                               effects

    vartype                  Description
        -------------------------------------------------------------------------
        independent              one variance parameter per random effect, all
                                   covariances zero; the default unless a factor
                                   variable is specified
        exchangeable             equal variances for random effects, and one
                                   common pairwise covariance
        identity                 equal variances for random effects, all
                                   covariances zero; the default for factor
                                   variables
        unstructured             all variances and covariances distinctly
                                   estimated

And I read online that COVTYPE(VC) requests the default (variance component) structure for random effects, which assumes all random effects are independent.

Improve this question
$\endgroup$
4
  • $\begingroup$ Just to clarify: which results are not consistent? The estimated parameters of the fixed effects (standlrt) and the estimates of the variance/covariance matrix of the random effects? Remember that by default Stata gives you the estimates of the standard deviations of the random effects. Are you sure you're comparing apples with apples? (It may sound like a stupid question, but it's good to rule out the simplest sources of error first) $\endgroup$
    – boscovich
    Apr 29, 2012 at 16:53
  • $\begingroup$ The last line of the SPSS code includes a particular structure of the random effects. I am not sure but this looks like SAS, where "VC" means that the random intercept is independent of the random effect "standlrt". Are you sure your STATA code includes such a structure too ? $\endgroup$
    – Stéphane Laurent
    Apr 29, 2012 at 17:32
  • $\begingroup$ Please post the full output from both packages. $\endgroup$
    – StasK
    Apr 29, 2012 at 17:49
  • $\begingroup$ By default in this case Stata uses a structure that "allows a distinct variance for each random effect within a random-effects equation and assumes that all covariances are zero". Stata calls it an independent structure, but afaik it's also known to be a variance components (VC) structure. So, it should be the same. $\endgroup$
    – boscovich
    Apr 29, 2012 at 18:35

1 Answer 1

Reset to default
7
$\begingroup$

Stata reports the estimated standard deviations of the random effects, whereas SPSS reports variances (this means you are not comparing apples with apples). If you square the results from Stata (or if you take the squared root of the results from SPSS), you will see that they are exactly the same.

For example, squaring the results from Stata:

.1214197 ^ 2 = .014742744 (standlrt)
.3032317 ^ 2 = .091949464 (Intercept)
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the answer! So simple yet so frustrating. Especially because the labels Std. Err. and Std. Error correspond. Also, the fixed effects results are identical when you look at the numbers. $\endgroup$
    – Raynor
    Apr 29, 2012 at 21:07
  • 4
    $\begingroup$ @C.Pieters The Stata postestimation command estat recov displays the random-effects covariance/correlation matrix, so that you don't have to square the results by hand (if you want the variances). $\endgroup$
    – boscovich
    Apr 29, 2012 at 21:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.