7
$\begingroup$

Imagine we were to investigate the relationship between people's annual income and daily food expenditure in a fictional population. The following example is not meant to be realistic, but hopefully serves to illustrate the point.

We define ten income groups: 100K, 200K, 300K etc. up to 1 million. For each group, we find 1000 people who have exactly those incomes and ask them how much they spend on food on an average day. We find the following distributions for each group (jitter is applied for better visualisation):

Income vs. expenditure

We calculate the mean and SD for each group. We then use simple linear regression and discover that there is a linear relationship between income and the means we found, and also a linear relationship between the income and the SDs (i.e. SD increases with increasing income).

We also discover that a log-normal distribution can be fitted for each group. This allows us to make a model that can predict the percentiles of expenditure for any income (at least within the range):

enter image description here

Imagine instead that we did not have access to those 10 neat income groups, but instead simply asked e.g. 600 random people (from the same population as before) about their income and their food expenditure, and found this:

enter image description here

Is it possible to approximate the percentiles shown in the second plot when the income variable isn't divided into discrete, equally sized groups? The residuals are heteroscedastic, and let's assume they are also log-normally distributed as before.

$\endgroup$
7
  • 3
    $\begingroup$ This looks like a use-case for quantile regression (slides by Roger Koenker). Also, have a look at p11 of the quantreg vignette ftp.uni-bayreuth.de/math/statlib/R/CRAN/doc/vignettes/quantreg/…. $\endgroup$
    – A.Fischer
    Dec 22, 2021 at 13:43
  • 1
    $\begingroup$ One of the simplest options is to account for heteroscedasticity by using something like a Huber-White estimator. Other options are to log-transform daily food expenditure or use a poisson/negative-binomial GLM. $\endgroup$ Dec 22, 2021 at 13:59
  • 1
    $\begingroup$ Interesting question, +1. I second the excellent suggestion of @A.Fischer. I believe question title could better reflect that you are not interested in linear regression, or predicting a mean and a variance (a Poisson regression would probably do a good job for that, like suggested by @philbo_baggins), but in modeling percentiles. $\endgroup$ Dec 22, 2021 at 14:04
  • 4
    $\begingroup$ If all you are interested in is percentiles, then this is indeed precisely quantile regression. You can use all the tricks of standard regression in this setup, like interactions, spline or other transformations. @A.Fischer: do you want to expand your comment into an answer? $\endgroup$ Dec 22, 2021 at 14:32
  • 3
    $\begingroup$ Thank you all! This is indeed very helpful. I love that the "Quantile regression in R" document" actually uses income vs. food expenditure as an example as well. Quantile regression definitely is what I am looking for, but actually poisson regression could prove useful to the project I'm working on as well. $\endgroup$ Dec 22, 2021 at 17:56

2 Answers 2

5
$\begingroup$

As you are interested in modeling percentiles, you should have a look at quantile regression methods. Instead of modeling conditional means (as in linear regression), quantile regression allows you to model (conditional) quantiles.

As mentioned in the comments, a good introduction to quantile regression is the vignette to the quantreg R package. One of the examples in the vignette illustrates your use case:

Quantile Regression, Example from the quantreg vignette

$\endgroup$
2
-1
$\begingroup$

In this case, it would make sense to try to predict the log of daily expenditure - this will probably be closer to linear and so easier to predict.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.