I'm working on a project where we want to estimate how many of a particular bird species there potentially could be in every 10-km square in England. For this I want to use quantile regression to get a 0.90 quantile estimate rather than the mean and so better estimate the possible abundance rather than the current abundance. However, I have count data as my dependent variable and all the quantile regression packages I can find in R seem to require normally distributed data. I also need a weight in my model to account for survey effort and again only some packages allow this.

I thought the qgam package might be the solution but that does not allow you to specify family and instead uses a family called "elf". I've tried reading about elf and found it's the extended log-F family, but I'm an ecologist not a statistician and I found myself lost in the equations!

So my questions are:

  1. Can anyone explain the elf family for me in a simple way?
  2. Is the elf family suitable for count data?

migrated from stackoverflow.com Mar 21 '18 at 11:51

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ Hi Jenni, have you considered using a probabilistic model (such as a GAM model based on Poisson family in the mgcv package), before trying qgam? Once you have fitted such a model, you'll be able to calculate any quantile simply using qpois(). qgam uses only the elf family, because this is a close approximation to the pinball loss, which is the loss typically used in quantile regression. You could make it work on count data by adding random jitter, but before doing that I would consider a Poisson model or similar. $\endgroup$ – Matteo Fasiolo Mar 23 '18 at 16:50

I don't think the elf distribution is suitable for modeling count data, but rather it is suitable for modeling continuous data. A nice description of the generalized log-F distribution is available on page 163 of the book Nonparametric Statistical Methods Using R by John Kloke and Joseph McKean. (I am presuming that the extended and the generalized log-F distribution are one and the same thing.)

In the earlier quantile regression literature for count data, people have used various tricks to translate their problem from count data to continuous data: log-transforming the count data and then applying quantile regression methods for continuous data to the log-transformed data, adding a quantity selected at random from a uniform distribution to each count to make it continuous and then proceed as one would for continuous quantile regression, etc. More recently, Bayesian methods for count data seemed to have gained traction as they provide increased flexibility.

In your own context, not only do you have count data for your outcome variable, but these data may exhibit spatial dependence given how they are collected. So that complicates things further. (Not sure temporal dependence is an issue - it will depend on when your data were collected - a single time point or multiple time points.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.