0
$\begingroup$

I am trying to predict the 10th, 30th, 50th, 70th and 90th quantiles of dependent variable. I have 110 independent variables in the data set. I can think of two approaches to do this.

  1. Use all data points and then predict above quantiles using quantile regression.
  2. First subset data into 5 groups i.e. [0-20], (20-40], (40-60],(60-80] and (80-100] and then predict median value for each group.

I am ultimately interested in coefficient values of independent variables at different quantiles.

With approach 1 - I mostly see either increasing or decreasing trend of coefficients from 10th quantile to 90 quantile.

I do not see such trend with second approach. However, what could be the potential pitfalls of second approach.

Improve this question
$\endgroup$
4
  • $\begingroup$ Have you tried doing some basic exploratory data analysis? Surely, there are a few variables that can be immediately dropped out from that process. I mean, you can at least look at some simple correlations. $\endgroup$
    – Jon
    Commented Jan 12, 2018 at 20:22
  • $\begingroup$ I have checked VIF and removed highly correlated features. 110 columns is after doing all necessary checks. $\endgroup$
    – Chandra
    Commented Jan 12, 2018 at 20:31
  • $\begingroup$ If you've checked VIF's, then I assume you've at least run 1 linear regression model. If you did, you probably saw the coefficients and p-values. I would inspect those with low effect sizes and/or high p-values to see if there is any significant relationship. If not, drop them. Model building is not an easy process; it's tedious work. $\endgroup$
    – Jon
    Commented Jan 12, 2018 at 21:21
  • $\begingroup$ @Jon, thank you for responding. My question does pertain to feature selection. It is regarding predicting different quantile values. I am seeking answer to identify whether predicting median after classification is same as predicting different quantiles using quantile regression. $\endgroup$
    – Chandra
    Commented Jan 12, 2018 at 21:25

1 Answer 1

Reset to default
2
$\begingroup$

Well, your second approach is subject to the pitfall of sample selection bias because in each regression of a subset, you literally drop the data points in the other subsets. Think about writing down the likelihood function of your subset regression cast as a maximum likelihood estimation problem. The likelihood should be conditioned on your dependent variable being within certain range.

Koenker, the inventor of QR, actually is careful to explain why QR is different from multiple regressions of subsets based on sorting the dependent variable. Check his introductory paper on the Journal of Economic Perspectives (pg 147): http://www.econ.uiuc.edu/~roger/research/rq/QRJEP.pdf

I am sympathetic toward your confusion as I had it too when first touching on QR. I encountered a similar question on a different QA website but could not help because I couldn't log on to that website. Just copy the question link here; https://www.researchgate.net/post/What_is_the_difference_between_running_a_quantile_regression_and_puting_the_dependent_variable_into_quantiles_and_running_multiple_regression_on_each Hopefully the author might be able to see this (better still he has solved the problem) or someone else who can use that website help answer that gentleman's confusion.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Good post. One minor thing - while Koenker did, indeed, write a seminal book and lots of papers on quantile regression, he didn't invent it. It has a long history going back to the 18th century. But Koenker did add a lot. $\endgroup$
    – Peter Flom
    Commented Oct 23, 2019 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.