I have one dependent variable (fish abundance) and one independet variable (time), both continuous. I would like to test the correlation between them because I expect that the abudance changes over time due to contextual factors. Analysing a ggplot using the "loess" method, I concluded that the two variables seems to have a quadratic polynomial behaviour. I would like to test the correlation between these two variables, but I don't know how to do that. I know that spearman is not an option (because the curve goes down and then up). I tried to use a nls model, which worked for making the curve, but I couldn't find any output (value) regarding the association between the two variables. I looked for a specific test, but I couldn't find. I read about the option of using a GAM and I tried it, but I'm not sure if I may use a GAM having only one indepent variable (since I always see GAMs with multiple variables, and looks like the model was created for that). Could I? If so, is the "deviance" on the GAM output an indicative of the relationship between the two variables? (something similar to the R2 for the linear relationships?)

Thank you very much in advance for your time.


You can answer your question to an extent using a GAM. It won't tell you about whether the relationship is quadratic or not, but it will estimate the shape of the relationship between x and y and you could use the deviance explained as a surrogate for $R^2$. However, I would not base my answer on the deviance explained, but on the results of the statistical test outputted by summary(), and I would want to make sure that the assumptions of the model were met by the data.

Here is an example with a non-linear relationship. There are data from an experiment that measured something (uptake) at a range of values of something else (conc) on a set of plants, each plant being measured at each conc.



## plot; the data look like this
ggplot(CO2, aes(x = conc, y = uptake)) +
  geom_point() +

enter image description here

## Model; x = conc, y = uptake
mod <- gam(uptake ~ s(conc, k = 6), data = CO2, method = "REML")

This gives:

> summary(mod)

Family: gaussian 
Link function: identity 

uptake ~ s(conc, k = 6)

Parametric coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  27.2131     0.9258   29.39   <2e-16 ***
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Approximate significance of smooth terms:
          edf Ref.df     F  p-value    
s(conc) 3.385  3.973 13.43 1.36e-08 ***
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

R-sq.(adj) =  0.384   Deviance explained =   41%
-REML = 299.08  Scale est. = 71.992    n = 84

The relationship appears significant and uses ~3 degrees of freedom — which is a little higher than we'd expect if the relationship were quaratic. However, if we look at the model diagnostic plots:


enter image description here

we see that the assumptions of this model are violated:

  • constant variance — the deviance residuals fan out as the value of uptake increases, and issue that is see in the other plots too, and
  • as mentioned earlier, we also ignored the fact that the data come from a small number of individual plants. Hence we don't have n = 84 unique data points as there is dependence or clustering at the level of the individual plant.

Accounting for those issues is beyond the scope of this Q&A, but we could model log(uptake) instead of uptake for example to deal with the heterogeneity, or use the Gamma family, and we would need some form of mixed or hierarchical model to deal with the dependencies at the plant level. A fuller attempt to model these data using GAMs is here.

Whilst you can use a GAM and the summary output to help answer the question you pose, you do need to ask the right question or at least understand what question the GAM is answering in this case), and you need to check that the model is appropriate for the data by doing the diagnostics.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.