5
$\begingroup$

In this blog article:

https://www.fharrell.com/post/improve-research/

it states:

“The frequentist paradigm does not provide confidence intervals or p-values when parameters are penalized”.

I was wondering how this applies to Generalised Additive Models (GAM’s). Standard software e.g. packages gam and mgcv in R provide confidence intervals, when shrinkage (penalised) estimation is used, even if we specify additional penalties (select=TRUE) in mgcv::gam().

How then are these confidence intervals valid?

$\endgroup$

1 Answer 1

6
$\begingroup$

This is completely fine in mgcv as it takes a Bayesian view of smoothing and as such automatically provides (empirical Bayesian) credible intervals and specifically-derived theory is used to give p values with asymptotically correct performance. (It so happens that if we take a frequentist interpretation of the credible interval we can interpret the coverage of that interval to be an average across the entire function.)

So, while Frank is correct to say that in general we don't have p values etc for regularized models (like the LASSO), this is not the case for GAMs fitted by mgcv. And increasingly it is also the case that we can produce valid p for models like the LASSO through new theory for post-selection inference, which take into account, in this case, the penalization that has been applied to the model parameters.

$\endgroup$
4
  • $\begingroup$ Thank you @Gavin Simpson. I think I grasp the usefulness of shrinkage for predicting VALUES of the response variable using the fitted model. But I'm finding it difficult to understand how such (sometimes heavy) shrinkage is valid for effect estimation i.e. difference in expected PARAMETERS (not values) of the response variable which can be computed using marginal/conditional means. Perhaps its the case that although the resulting marginal means will be biased, the confidence intervals are still valid for this biased point estimate, if that makes sense? $\endgroup$
    – user167591
    Commented May 29 at 10:03
  • 1
    $\begingroup$ In models like the Lasso, we trade off a little bias for a hopefully larger reduction in the variance which reduces the overall RMSEP. The same thing happens in the estimates of $\beta_k$ for the basis functions of penalized splines in a GAM. Additionally, if for example there is correlation between covariates, this bias can also extend to the unpenalized terms in the model. This suggests that choosing smoothing parameters via REML might be the most appropriate method as it tends to undersmooth slightly compare to GCV selection. $\endgroup$ Commented Jun 3 at 8:10
  • 1
    $\begingroup$ The confidence intervals are in general OK, and their frequentist interpretation tends to hold so long as the bias in the $\hat{\beta}_k$ (and hence the estimated smooth) is small relative to the variance. See Marra & Wood (2012) for the details: Marra, G., Wood, S.N., 2012. Coverage Properties of Confidence Intervals for Generalized Additive Model Components. Scand. Stat. Theory Appl. 39, 53–74. doi.org/10.1111/j.1467-9469.2011.00760.x $\endgroup$ Commented Jun 3 at 8:12
  • $\begingroup$ Thank you @Gavin Simpson, for your answer and reference $\endgroup$
    – user167591
    Commented Jun 5 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.