11
$\begingroup$

I am using the gam function (from the mgcv package) to model a continuous response (a soil nutrient) in relation to a continuous predictor plus categorical predictors plus site random effects (modest number of sites) with:

    gam(log(y) ~ s(x, bs = "cr", by = fac1) + 
              fac1 + fac2 + s(site, bs = "re", by = flag), 
        method = "REML")

The data come from a landscape study where sampling was challenging (to say the least), so the number of samples is small and the coverage of the range of $x$ values is quite patchy in the upper part of the range.

Use of a cubic spline for $x$, as above, results in quite a different shape for the fitted function compared to the default thin-plate spline for that part of the space where $x$ values are sparse. However, both models have almost identical results in terms of deviance explained and pattern of residuals. The shape of the function produced by the cubic spline is much preferred by my colleagues who conducted the study because they feel they can reconcile it with theory more readily than the alternative. However, I am wondering if:

  • There are other criteria that I can investigate to choose between the models?
  • Any theoretical reason to choose between using "cr" or "tp" for a univariate smooth with patchy data?

Update: plots of the alternative models

GAM using bs = "cr"

GAM using bs = "tp"

$\endgroup$
6
  • 1
    $\begingroup$ Can you include PNGs of the two fitted functions? Thin-plate splines have the edge in terms of MSE over the cubic regression spline, but are much more costly to set up the basis function for. I'd be interested to see whether the effects of the two splines are really that different given the uncertainty in their estimation at end points of a variable, esp if the data is patchy. $\endgroup$ Commented Jan 24, 2017 at 19:13
  • 2
    $\begingroup$ Finally, without any further info, it would be cherry picking to choose the type of spline basis (i.e. the model) simply because your colleagues prefer one output better than the other. If you end up having no objective reason to select one basis type over the other and your models are otherwise OK, you have a responsibility to present both and to proceed with interpreting both. It would be technically unsound and ethically dubious to cherry pick a model over another just because it gave you/them results they preferred. $\endgroup$ Commented Jan 24, 2017 at 19:21
  • 1
    $\begingroup$ @GavinSimpson - Thanks. I've added plots showing predictions from the two models. The two sets of residuals are almost identical and seem well-behaved as judged with qqplot and graphing against fitted values and predictors. gam.check showed that EDFs were well below k' values. Unfortunately, the x variable has only a small number of unique values (18) so there's not much room for adjustment above the default k=10. I agree with your view that the best course is to present both results, assuming both survive scrutiny here. $\endgroup$
    – michael
    Commented Jan 25, 2017 at 6:23
  • 1
    $\begingroup$ After canvassing opinions from a number of others, it's been decided to go with the thin-plate regression spline fit on the basis (no pun intended) of parsimony. $\endgroup$
    – michael
    Commented Jan 31, 2017 at 5:38
  • 1
    $\begingroup$ Yup, the CRS fit for group 1 are crazy; there seems no reason for them to take a unimodal shape. $\endgroup$ Commented Jan 31, 2017 at 5:43

1 Answer 1

0
$\begingroup$

Quite an old question that seemed to be mostly answered in the comments already. But in any case, it seems clear that the thin plate regression splines are the clear contender here. The curves on the cubic regression splines make no sense and have a ridiculous standard error region.

Something to add beyond that...there really aren't a lot of observations here, which contributes to the problem greatly. A future design would undoubtedly be better with even a little more data sprinkled in. Its hard to understand exactly what the actual function relationship is when you have such extreme values on one end and a clearly flat association with lower values. The real relationship may actually have a much stronger exponential rise with increases in $x$. But without more data, that remains totally unclear.

A full Bayesian implementation may be better in this respect. Incorporating prior information when fitting the curves (such as those possible with brms or bamlss) would do wonders to show what a more plausible fit should be given how few datapoints there are.

$\endgroup$
2
  • $\begingroup$ I'm a big fan of brms but don't see how prior information could have been incorporated here. This study was on the effects of time since different types of fire in naturally vegetated areas, and the distribution of times in the study area was very spotty. My first resort these days would be to constrain k values on smooth terms. I probably would fit a model in a Bayesian manner, or do posterior simulation from a model fitted with mgcv, to quantify predicted differences between groups etc. But if you can point to examples of using priors in this regard I'd be very interested. $\endgroup$
    – michael
    Commented May 8 at 2:15
  • $\begingroup$ Ah perhaps that wouldn't be helpful here then...that was just my first thought reading this question. Otherwise I think the thin plate splines aren't wrong. They do the best job considering. $\endgroup$ Commented May 8 at 5:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.