In a linear model including ANOVA one can test a trend (e.g., linearity, quadratic effect, etc.) among the ordered effects (regression coefficients or factor levels) through assigning proper weights. For example, suppose that we have a time series regression model


where $Y$ is a vector of length $n$, $X_i$ is an explanatory variable of length $n$ ($i=1,2,3,4$), and $\epsilon$ is the residual vector. The linear and quadratic trends among four equally-spaced (and ordered) effects $\beta_1, \beta_2, \beta_3, \beta_4$ can be tested with weights of -3, -1, 1, 3, and 1, -1, -1, 1 respectively. When those effects are not equally-spaced, such weights can be obtained through the coefficients of orthogonal polynomials.

My understanding is that typically the exponential trend among some sequential effects can be tested through log-transforming the response variable. However, the situation I'm dealing is not directly about the response variable, but the effect estimates (or regression coefficients, e.g., 30 ordered effects similar to a linear model above). So the challenge I'm facing here is, how to find the weights when testing for an exponential trend, when the effects are equally- or unequally-spaced?

P.S. There are two time scales involved here that may cause some confusions. At the local time level, each explanatory variable $X_i$ in the time series regression model codes for a causal event that lasts for a short period of time. At the higher level of time, there is the sequence of those causal events and their estimated effects $\beta_1$, $\beta_2$, .... It is the exponential trend or saturation effect among those estimated effects that is of interest to me. For a linear trend, I could perform a general linear hypothesis testing with appropriate weights for those effects. However, I'm not so sure whether it's feasible to do the same for an exponential trend (or saturation effect).

  • $\begingroup$ Can you more clearly explain the situation please? It sounds like you have some ordered categorical explanatory variable which you want to test for no trend vs an exponential trend, is that right? You wouldn't normally do that by setting up some set of contrast coefficients (weights), but more typically by other means. $\endgroup$ – Glen_b -Reinstate Monica Mar 22 '14 at 0:54
  • $\begingroup$ @Glen_b: Thanks for the suggestion. I've modified my question in the original post. See if it's clearer now. $\endgroup$ – bluepole Mar 22 '14 at 17:10

The basic problem I see is that an exponential trend is not a linear function of parameters.

If we're dealing with a nonlinear function of parameters, it looks to me like you may be better placed to deal with it either in the context of a glm (if possible), or perhaps a nonlinear least squares model.

So if the null is an exponential progression of coefficients, you could write that as a model that has that form, (while the alternative is of the general form of your original model), and fit them both, then test the difference (the models are effectively nested).

An example of the sort of analysis I am talking about, done in R:

x <- c(3L, 4L, 4L, 5L, 3L, 2L, 4L, 5L, 1L, 2L, 1L, 1L, 2L, 3L, 1L, 
3L, 5L, 3L, 3L, 1L, 1L, 2L, 2L, 4L, 5L, 2L, 5L, 3L, 1L, 1L, 2L, 
3L, 5L, 2L, 3L, 5L, 4L, 3L, 4L, 4L, 5L, 2L, 1L, 3L, 4L, 3L, 5L, 
1L, 2L, 2L, 5L, 5L, 2L, 1L, 4L, 1L, 4L, 4L, 2L, 5L)

y <- c(3.505, 3.432, 1.794, 2.952, 8.19, 5.38, 3.62, 4.474, 12.114, 
6.375, 9.835, 10.734, 7.102, 6.618, 12.779, 5.473, 3.305, 5.476, 
4.806, 9.874, 10.324, 5.762, 8.267, 2.914, 1.696, 8.91, 2.061, 
5.619, 9.659, 11.209, 5.617, 4.809, 3.799, 5.052, 6.953, 3.008, 
1.236, 5.362, 3.054, 4.123, 0.857, 7.095, 11.042, 4.873, 3.157, 
4.769, 1.174, 11.938, 7.386, 7.346, 3.352, 1.266, 9.078, 8.583, 
4.254, 10.088, 2.727, 4.67, 4.796, 1.607)

nefit <- nls(y~a*exp(-b*x),start=list(a=15,b=-.4))
facfit <- lm(y~-1+as.factor(x))

Analysis of Variance Table

Model 1: y ~ a * exp(-b * x)
Model 2: y ~ -1 + as.factor(x)
  Res.Df Res.Sum Sq Df Sum Sq F value Pr(>F)
1     58     89.643                         
2     55     82.108  3 7.5348  1.6824 0.1814

This indicates that the factor model doesn't add anything substantive over the exponential decay model, since the sum of squares contributed by the extra degrees of freedom isn't distinguishable from noise (and it shouldn't be, since I generated the data from the exponential model).

I said earlier that these models were "effectively nested".

To make them explicitly nested, Model 2 above could be parameterized as a nonlinear least squares model, such as $E(y) = \alpha \, \exp(-\beta x) + \delta_3 f_3 + \delta_4 f_4 + \delta_5 f_5$ - where the $f$'s are dummy indicators of group membership (in levels of the factor). If the model converges, it has the same fit as the linear model, and then the test of $\delta_3=\delta_4=\delta_5=0$ is the same as the one in the anova above (df=(3,55), sum of squares=7.5348, F=1.6824, p-value=0.1814). The approach I used above is equivalent but easier.

  • $\begingroup$ Thanks for the suggestion. Suppose the exponential relationship is f(t) = a * exp(-bt). Using the Taylor expansion, we have f(t) = a [1 - bt + (bt)^2 / 2! - (bt)^3 / 3! + ...]. Maybe I could try modeling the polynomial effects of, for example, up to the 5th order as a reasonable approximation for the exponential relation. I use the coefficients of the orthogonal polynomials as weights for a general linear testing in the linear model. In the end I would have five effects estimates that are associated with t, t^2, .., and t^5. The problem is, how can I retrieve a and b in f(t) = a * exp(-bt)? $\endgroup$ – bluepole Mar 24 '14 at 16:34
  • $\begingroup$ My answer above explains how to get $a$ and $b$ in $f(t) = a \exp(-bt)$ $\endgroup$ – Glen_b -Reinstate Monica Mar 25 '14 at 0:50
  • $\begingroup$ Sorry I'm still lost. Could you elaborate a little bit more. The limitation is that the trend analysis would have to be performed among those $\beta$ values in the time series regression model in my OP. $\endgroup$ – bluepole Mar 25 '14 at 18:47
  • $\begingroup$ Sorry, your comment is unclear. Your model seems to ignore the times series aspect, there's no lagged variables or anything, so I assume your model is just a straight regression even if the data are observed over time. Do you mean your $X$'s are lagged values of (something) and you're testing for a particular form of time series model? If so, that would utterly change my answer (and I'd be astonished that you would imagine that that critical information could be omitted and not alter the answer). If that is the case, what is the lagged thing? Is it just lags of the $Y$? $\endgroup$ – Glen_b -Reinstate Monica Mar 25 '14 at 21:36
  • 1
    $\begingroup$ See updated post. $\endgroup$ – Glen_b -Reinstate Monica Apr 2 '14 at 7:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.