2
$\begingroup$

Does having a set of predictors which are not linearly separable slow down the model fit in STAN? If so, why?

I have tried to test this, and it appears to slow down the fit. I fit a model with 10,000 rows and 61 linearly separable columns except for the intercept, 100 iterations for two chains and it finished in 7 minutes.

I fit a model with 61 columns which had columns which were linear combinations and it has taken over 18 minutes to finish the first chain.

$\endgroup$
2
  • $\begingroup$ Can you post code for a reproducible example? $\endgroup$ – Ellis Valentiner Mar 24 '15 at 20:47
  • $\begingroup$ I can try to come up with an example, but I cannot share the data that I am working with so will take a few. $\endgroup$ – goldisfine Mar 24 '15 at 20:49
1
$\begingroup$

Yes, it does. I think you are introducing multicollinearity and your model is not identified. Stan is not very happy with non-identifiability. I know that you can solve not-perfect multicollinearity with tight normal or cauchy priors but I think you will need hard priors for perfect multicollinearity. If one of your predictors is a linear combination of the other 60. You can exclude that one which is basically a hard prior of 0 on the coefficient for that predictor.

$\endgroup$
2
  • 1
    $\begingroup$ Keep in mind that the adding priors will make the model identifiable in a technical sense, but in the presence of perfect multicolinearity one must wonder what, precisely, your priors have accomplished since the data contributes precisely nothing. The stan manual has a great example for a model with two intercepts. $\endgroup$ – Sycorax Jul 31 '15 at 5:02
  • $\begingroup$ You are absolutely right. My comments about the priors should be read as: there is a connection between adding soft constraints with imperfect multicollinearity and excluding a variable because it does not contain any new information with perfect multicollinearity. It is not meant as advice to use hard constraints as a default prior when you know you have multicollinearity. $\endgroup$ – stijn Jul 31 '15 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.