Linear regression and prediction on transformed data

Question

I have a 7 points dataset that looks like this:

dates <- as.Date(sprintf('2011-01-%s', 15:21))
vals  <- c(0.05816136, 0.31020050, 0.45542547, 0.52401006, 0.56796386, 0.59687058, 0.61741189)

dat <- data.frame(date=dates, val=vals)

plot(dat$date, dat$val, type='b')

The plot is given here.

I'd like to create a prediction based on this dataset and I approach it so:

transf   <- function(orig) 170 ** orig
untransf <- function(trnsf) log(trnsf, base=170)

# transform the dependent variable to linearity and do a linear fit
fit <- lm(I(transf(val)) ~ date, data=dat)

test.dates <- as.Date(sprintf('2011-01-%s', 15:27))
plot(test.dates, untransf(predict(fit, newdata=list(date=test.dates))), type='b')
points(dat$date, dat$val, col='red', type='b')

However, as the second plot suggests the true values (red) follow slightly different shape than the so predicted ones. What's the problem with the approach I'm taking? Is there a general problem in transforming the data using an exponential function and then using the inverted prediction?

Answer 1

There are a number of issues with such a strong transformation.

1) Means don't carry across nonlinear transformations (that is, $\text{E}[\varphi(x)] \neq \varphi[\text{E}(x)]$; indeed, for a convex function, $\varphi\left(\text{E}\left[X\right]\right) \leq \text{E}\left[\varphi(X)\right]$). In some cases the bias can be severe.

2) by ignoring the implied effect of the transformation on variance, the relative weights to observations can differ quite markedly (equal weights on the transformed scale imply unequal weights on the untransformed scale); since your model doesn't fit all that well given the smoothness of the data, the lack of fit to the relatively high-variance points can become strong.

---

My suggestion is to try a nonlinear least squares model to the original data.

You should incorporate any features you understand about the process into the model. For example if you expect it to have a horizontal asymptote eventually, you can incorporate that understanding; this kind of information is critical to reasonable behavior beyond the end of the sampling period. (For example, if you know that 1 is an upper bound, that information would be important.)

For example, here's an asymptotic exponential fit ($\text{E}(y) = \beta_0 + \beta_1 e^{-\beta_2 x}$):

x <- 1:7
valfit <- nls(dat$val~b0+b1*exp(-b2*x),start=list(b0=.7,b1=-1,b2=1))
points(dat$date,fitted(valfit),col=6,type="b")

It's not a perfect fit (I wouldn't want to extrapolate more than a day or so), but it's a good deal better than the fit in your plot (which is different data). Note that since you (presumably) chose the "170" based on the data, you effectively have three parameters there.

If you do use that particular model above, note that it has a self-start function (I think it's called SSAsymp), so you don't need to supply starting values as I did (I chose those values simply by looking at the plot of the data).

If you thought the function might eventually be linear, you might add a linear term to that asymptotic model, which produces predictions like so:

But with only 7 points so many parameters is hard to justify.

You may (and hopefully do) have a logical reason to choose a better function than mine, of course (and any well justified choice should certainly be used over what I have suggested, which is merely an example) -- but on the other hand I seriously doubt you have any good justification for taking 170th powers.

---

As @FrankHarrell suggests a regression spline on the untransformed data is well worth considering. In that case I'd probably suggest a natural regression spline (with low degrees of freedom).

Such an approach has a number of advantages.

Note that natural splines will be linear beyond the ends of the data, so out of sample predictions from a well-fitting model will be straight lines whose slope is close to the general trend at the end of the observed data.

Here's an example, which shows that 3df may (slightly) underfit, while 4 may overfit:

 library(splines)  # comes with R
 with(dat, {
   valnsfit3 = lm(val ~ ns(x, df=3))
   valnsfit4 = lm(val ~ ns(x, df=4))
   plot(date,   val, type='b')
   points(date, fitted(valnsfit3), col=3, type="l")
   points(date, fitted(valnsfit4), col=4, type="l")
})

Given such low sample size I'd probably have just used the smaller model, though if you had more data (around say 10 points) I'd probably go with 4df.

The code above generates the plot below; the fit in green has 3 df and the blue one has 4 df.