4 Added one more bullet point describing an aspect of the issue
source | link
  • Running stepwise at tau = 0.9 produces a final model with 7 variables and AIC in the neighborhood of 16,000. The model looks pretty reasonable from the perspective of the phenomenon we're studying. However, running the same at tau = 0.99 produces a monster model that includes almost 90% of our variables, a lot of them with bizarre giant coefficients, & AIC at something like 45,000. Why would a change in tau lead to such a dramatic change in results?
  • In a previous version of the data, which had about 70 variables, tau = 0.9 would run (yielding the SAME parsimonious model as in the bullet point above), whereas tau = 0.99 would fail due to singularity in the matrix. We "resolved" this problem by paring down our potential predictor variables -- though we did this on the basis of theory, not because anything about them was obviously liable to lead to matrix singularity (they were not involved in collinearity or multiples of one another or anything like that). Is there a general reason why a higher tau and/or higher number of predictor variables would lead to increased likelihood of singularity?
  • For all cases, R will report some warning message like "973 non-positive fis". The number is higher for higher tau & more variables.
  • Running stepwise at tau = 0.9 produces a final model with 7 variables and AIC in the neighborhood of 16,000. The model looks pretty reasonable from the perspective of the phenomenon we're studying. However, running the same at tau = 0.99 produces a monster model that includes almost 90% of our variables, a lot of them with bizarre giant coefficients, & AIC at something like 45,000. Why would a change in tau lead to such a dramatic change in results?
  • In a previous version of the data, which had about 70 variables, tau = 0.9 would run (yielding the SAME parsimonious model as in the bullet point above), whereas tau = 0.99 would fail due to singularity in the matrix. We "resolved" this problem by paring down our potential predictor variables -- though we did this on the basis of theory, not because anything about them was obviously liable to lead to matrix singularity (they were not involved in collinearity or multiples of one another or anything like that). Is there a general reason why a higher tau and/or higher number of predictor variables would lead to increased likelihood of singularity?
  • Running stepwise at tau = 0.9 produces a final model with 7 variables and AIC in the neighborhood of 16,000. The model looks pretty reasonable from the perspective of the phenomenon we're studying. However, running the same at tau = 0.99 produces a monster model that includes almost 90% of our variables, a lot of them with bizarre giant coefficients, & AIC at something like 45,000. Why would a change in tau lead to such a dramatic change in results?
  • In a previous version of the data, which had about 70 variables, tau = 0.9 would run (yielding the SAME parsimonious model as in the bullet point above), whereas tau = 0.99 would fail due to singularity in the matrix. We "resolved" this problem by paring down our potential predictor variables -- though we did this on the basis of theory, not because anything about them was obviously liable to lead to matrix singularity (they were not involved in collinearity or multiples of one another or anything like that). Is there a general reason why a higher tau and/or higher number of predictor variables would lead to increased likelihood of singularity?
  • For all cases, R will report some warning message like "973 non-positive fis". The number is higher for higher tau & more variables.
3 Fixed wrong phrasing in last sentence
source | link

I suspect one or several of the characteristics of our data is leading to the strange outputs (radically different at tau = 0.9 vs. 0.99). Is there a conceptual reason for this & how might we be able to address it?? Do any of the red flagsdata characteristics described clearly invalidate the approach here?

I suspect one or several of the characteristics of our data is leading to the strange outputs (radically different at tau = 0.9 vs. 0.99). Is there a conceptual reason for this & how might we be able to address it?? Do any of the red flags described clearly invalidate the approach here?

I suspect one or several of the characteristics of our data is leading to the strange outputs (radically different at tau = 0.9 vs. 0.99). Is there a conceptual reason for this & how might we be able to address it?? Do any of the data characteristics described clearly invalidate the approach here?

2 Added very brief R code, in the unlikely event my issue actually lies there
source | link

The code is essentially just:

stepfit <- step(rq(response ~ ., data = data, tau = 0.99), direction = "both")

Here are the main red flags I'm seeing (though they raise somewhat separate questions, I'm presenting them all, since I wonder if the issues are in fact related):

Here are the main red flags I'm seeing (though they raise somewhat separate questions, I'm presenting them all, since I wonder if the issues are in fact related):

The code is essentially just:

stepfit <- step(rq(response ~ ., data = data, tau = 0.99), direction = "both")

Here are the main red flags I'm seeing (though they raise somewhat separate questions, I'm presenting them all, since I wonder if the issues are in fact related):

1
source | link