4
$\begingroup$

I am facing a modeling problem:

$t_{ij} = D_i + T_j + \epsilon_{ij}, i=0...641, j\in\mathbb{N}$

where $\epsilon_{ij}$ follows exponential distribution,

$\epsilon_{ij} \sim \lambda e^{-\lambda t}, \lambda \approx 1/26 \mathrm{ns}$.


If $\epsilon_{ij}$ is treated as normal, the problem can be modeled as ANCOVA. For example in R:

# i and j are categorical (factors in R)
lm(t ~ i + j, data)

It works as a good approximation, but the residual plot is highly skewed. I am thinking of treating $\epsilon_{ij}$ as exponential.

In terms of a generalized linear model, I need a shifted exponential as residual distribution. The questions are:

  1. Is a shifted exponential distribution in the exponential family?
  2. If 1 is yes, how can I express it as an R glm() call? glm can use poisson, binomial, etc. as residual distributions. But no exponential is provided.
  3. If 1 is no, what is the best way to fit this model? Should it be generalized nonlinear model (R package gnm) or something else?

I wrote a maximum likelihood fitter for the exponential residual, but it is slow and non-intuitive.

$\endgroup$
  • $\begingroup$ When you say "ANCONA" what does the second N stand for? Are $D$ and $T$ the parameters in your model or are they predictors? If this is a model with factors in both variables, what does the "C" stand for? $\endgroup$ – Glen_b Jul 18 '16 at 5:57
  • $\begingroup$ "ANCONA" was a typo. It's analysis of covariance or ANCOVA. $D$ and $T$ are parameters. $i$ and $j$ are predictors. "C" is for Covariance. $\endgroup$ – heroxbd Jul 19 '16 at 1:41
  • $\begingroup$ $j \in \mathbb{N}$? So you have an infinite number of samples? $\endgroup$ – Cliff AB Jul 19 '16 at 4:34
  • $\begingroup$ I have a finite number of samples. But the number of samples (data size) does not have an upper limit. $\endgroup$ – heroxbd Jul 19 '16 at 5:22
4
$\begingroup$

GLMs don't model additive errors, they model conditional distributions* -- for an exponential model you have $(Y|\mathbf{x})\sim \text{Exp}(g^{-1}(\mathbf{x}\beta))$ (here using the scale-parameterization of the exponential not the rate-parameterization).

plot of model with additive exponential error vs plot of conditional exponential model

The first model has constant variance, the second has variance that's a function of the mean.

The question you link to is asking about a conditionally exponential model (i.e. a GLM) not a shifted-exponential model (additive error model)

Is a shifted exponential distribution in the exponential family?

No.

how can I express it as an R glm() call? glm can use poisson, binomial, etc. as residual distributions. But no exponential is provided.

The exponential is a special case of the Gamma and is easily done in R ... but it's not shifted exponential

If 1 is no, what is the best way to fit this model? Should it be generalized nonlinear model (R package gnm) or something else?

gnm also assumes exponential family

I don't think MLE is difficult$^\dagger$; if you find the highest hyperplane that has no points below it, it should maximize the likelihood for the location parameters.

One way to find that is via quantile regression (such as the function rq in the package quantreg in R, by seting tau=0)

You can then get a sensible estimate of scale from the mean of the non-zero residuals from that fit (though the ML estimate would include the zero residuals, which is fine, though it will be biased of course). Adding that mean to the intercept from the quantile regression should give the intercept-estimate you were after.

For a pure ANOVA type model this could be overkill (in the sense of there being faster ways to identify parameters) but it seems to work quite nicely on what I tried it on.


* at least not in general; the subset of GLMs that are also LMs being the obvious exception

$^\dagger$ As long as you don't fall for trying to set derivatives to 0 ... just plugging this straight into some vanilla optimizer without a bit of thought first is not likely to work very well in general.

$\endgroup$
  • $\begingroup$ Thank you very much. I have used rq() from quantreg and it works great. However, rq() is not quite scalable. With a 100MB dataset, the memory footprint could be as large as 64GB. What's your recommendation to do quantile regression on a larger dataset? $\endgroup$ – heroxbd Nov 17 '16 at 9:46
  • 1
    $\begingroup$ Oh .... that's a whole new bucket of issues. It might be best to ask as a new question (with more details about the exact model) as someone else may be better at answering that, but going back to framing it as "find the highest hyperplane that has no points below it" should be doable via faster methods. $\endgroup$ – Glen_b Nov 17 '16 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.