1
$\begingroup$

I've had such a hard time finding a source that addresses how to handle multicollinearity of parametric terms in a generalized additive model (GAM) that I'm starting to think it may not be important, but why might this be? Multicollinearity is a big issue in linear regression, but it seems to be hardly mentioned when a GAM is involved. I am aware of vif.gam(), but unfortunately it does not handle factors, or interactions involving them, so I'm left to assume it's a none issue. I have also tried the function concurvity(), but this lumps all parametric terms into a single group and I have failed to find a remedy for high concurvity between smooth terms and the grouped term "para". As long as the model runs, can the highly (correlated?) relationship between a parametric version of a factor and a continuous variable (for example) really be ignored in a GAM(M)? Is it less influential in a given GAM (model) then the smooth terms? All corrections of any misunderstanding I have of the issue are welcome.

$\endgroup$

1 Answer 1

2
$\begingroup$

Multicollinearity can always be a problem, also in GAMs.

First, if you do not specify smooth terms in your GAM model, then it is basically a linear model.

Further, suppose you have two predictors that only differ by random noise, then it is fundamentally hard or impossible to tell to what extend each predictor is to be associated with the response. It strongly depends on the random noise by which the two predictors differ.

Many combination of the two predictors seem plausable.

This leads to unstable calculations in any algorithm and parameter or smooth estimates that can differ a lot even if your data varies only a little.

Try it by yourself by generating collinear data and fitting a GAM model

$\endgroup$
3
  • $\begingroup$ Hi Chris, thank you. My model has a linear interaction term of a factor (2 levels) * 1 continuous variable + 3 smooth variables. The interaction is a temporal term and the smooths are environmental covariates (which vary in time). I was told to include them all to make sure all effects are accounted for, but naturally they're closely correlated. Ignoring this specific time-dependent example though (I'm hoping for a more general method of resolving an issue like this), do you know of a method for dealing with related linear and smooth covariates? $\endgroup$
    – Nate
    Oct 27, 2023 at 14:07
  • 1
    $\begingroup$ Hi Nate, I don't know of such a method. But you have a small number of predictors. You might compare model performance of subsets of your predictors where you exclude the correllated ones on unseen data. If the full model performes best then you can go with it. Another solution might to find a way to combine some of the correlated predictors into a smaller set of predictors. There are some methods availabe to do that such as pca. $\endgroup$
    – ChrisL
    Oct 28, 2023 at 8:35
  • $\begingroup$ I see, thank you! $\endgroup$
    – Nate
    Oct 28, 2023 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.