1
$\begingroup$

I'm trying to fit a linear model to describe student performance in 2 different schools.

My response variable is $$Y_{ij}= X_{ij}*\beta+Z_{ij}*\gamma_j + \epsilon_{ij}$$ .

$$i = 1,...,n $$ $$j = 1,2 $$

$\beta $ are the fixed parameters, while $\gamma_j$ is the vector of mixed ones.

$\epsilon_{ij}$ ~ $N(0,\sigma^2)$

$\sigma^2$ ~ $Inv-Gamma(a,b)$

$\beta$ ~ $N_p(b_0,B_0)$

$\gamma_j$ ~ $N_p(0,\Sigma)$

$\Sigma$ ~ $Inverse-Wishart(S_0^{-1},\eta_0)$ .

In my model $X_{ij}$ and $Z_{ij}$ are the same, so that to each predictor is associated a fixed and a random effect.

I've tried to implement this model both with MCMChregress function in R and $Jags$, but unfortunately I can't find any literature regarding the parameters of my mixed effects (for $\beta$ I used an uninformative prior and then I cheated using the ols estimates, but I still get no significance in my mixed effects ).

$\endgroup$
4
  • $\begingroup$ What do you mean by X and Z being the same? If you have only two schools, the random effect is about with what exactly? It sounds rather like a fixed effects regression performance = b0 + b1 * predictor + b2 * school1_dummy... $\endgroup$ – Tim Jul 20 '16 at 10:10
  • $\begingroup$ Why fixed effect? I'm considering a model like: $performance_i$ = $\beta_0+\gamma_0+(\beta_{sex}+\gamma_{sex})*sex_i+(\beta_{sports}+\gamma_{sports})*sports_i$, The $\gamma$ change according to the school one belongs to, so they are mixed effects. $\endgroup$ – Tommaso Guerrini Jul 20 '16 at 12:43
  • $\begingroup$ If so, then please edit to clarify what exactly is your data and your model, since from the initial description it sounded as you are calculating something totally different... $\endgroup$ – Tim Jul 20 '16 at 12:57
  • $\begingroup$ Where is the problem? Both $\beta$ and $\gamma$ are distributed as a multivariate Normal of dimension p. $\endgroup$ – Tommaso Guerrini Jul 20 '16 at 12:59
2
$\begingroup$

Part of your issue could be that you only have two schools, so there's not much information at the school level. (You're trying to determine gamma with essentially two data points.) I've read that as a rule of thumb you want five or more groups, and your non-significance could simply be that you don't have enough information to nail down your coefficients.

You could add strong priors, but then the priors will be driving things. If you have principled priors, that could be an option, otherwise not.

If you are running into convergence problems -- you have checked for proper convergence, right -- weak priors (say a wide Normal) can help because uniform priors throw a ridiculous amount of weight to +/- infinity, which is probably not reasonable.

I'd also recommend using the brms package in R. It uses Stan under the hood and along with rstanarm is the cutting edge of Bayesian modeling in R.

EDIT: In terms of the number of groups required for good group-level estimates, see papers like: http://joophox.net/publist/methodology05.pdf and I'm also pretty sure it's mentioned by Gelman, but can't find it right now.

$\endgroup$
6
  • $\begingroup$ Thanks for answering. I have n data points, 2 $p$ dimensional vectors ($\gamma$). I do have good diagnostics for convergence to the stationary distribution of both $\beta$ and $\gamma$ , but the last ones didn't make sense. This question was posted months ago, in the meanwhile I partially solved the issue over the prior for $\Sigma$ , after having performed a variable selection and tried different $S_0$ matrices. My question was: do you know any literature regarding the $S_0$ hyperparameter? thank you again $\endgroup$ – Tommaso Guerrini Jul 20 '16 at 13:26
  • $\begingroup$ @TommasoGuerrini: Sorry, I don't know much about that. (I still think that having only two schools will make estimates for $\gamma$ more highly uncertain and hence much more likely to be non-significant.) $\endgroup$ – Wayne Jul 20 '16 at 14:51
  • $\begingroup$ The 2 vectors are sampled through a Gibbs sampler, hence increasing the number of schools would harden the MCMC $\endgroup$ – Tommaso Guerrini Jul 20 '16 at 15:22
  • $\begingroup$ Just to make sure we're on the same page, group-level parameters are being fit in your case with only two data points: the two groups. It doesn't matter how many data points you have within each school if you are trying to fit something at the level of between-schools. Perhaps you're not doing that and I'm just confused, but when trying to fit a distribution of multiple schools, it's the $j$ that matters, not the $n$. $\endgroup$ – Wayne Jul 20 '16 at 17:49
  • 1
    $\begingroup$ Please see my EDIT, above. If you are looking at group-level estimates with few groups, there will be problems. Again, I may be mistaking what's happening in your model, but to the extent you're having problems with group-level coefficients, I believe this is the key. $\endgroup$ – Wayne Jul 21 '16 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.