1
$\begingroup$

I'm modelling a species' response to environmental variables while controlling for spatial autocorrelation and temporal differences in sampling. The goal is to use this model for prediction.

I've been testing the predicitve power of various models by cross validation (train model with 80% of data, predictions on remainder). I initially tried using gam() from the mgcv package but found that the predictions weren't very accurate. I then tried using fitme from the spaMM package and found that this made more accurate predictions (code and example data below).

Why are the predictions from gam less accurate than spaMM? I wonder if I've misspecified the gam model? The diagnostic plots look fine for the gam model (less so for the spaMM model - code below). The only difference in these models is the smooth for the interaction between week, lat and lon - I don't know what the equivalent for this would be in the spaMM model (any ideas?).

Is it just that spaMM is a more appropriate approach with this data? My preference would be to use gam as I find the results easier to understand, although I'd be happy to use spaMM (or any other approach for that matter) if it gives better predictions (i.e. if the difference in predicitve power I've found is real and not due to errors in my approach).

Code:

    library(mgcv)
    library(spaMM)
    
    # function to calculate root mean squared error
    RMSE <- function(f, o){
      sqrt(mean((f - o)^2))
    }
    
    # set up training and validation sets
    set.seed(333)
    fold <- sample(seq_len(nrow(df)),size = floor(0.8*nrow(df)))
    train <- df[fold,]
    validate <- df[-fold,]
    
    # GAM
    m.gam <- gam(species_obs ~ 
                 + temp
                 + rainfall
                 + s(lat, lon, k = 50, m = c(1, 0.5))
                 + s(week, k = 7)
                 + ti(lat, lon, week, d = c(2,1), bs = c('ds'), m = list(c(1, 0.5), NA), k = c(20, 7))
                 + offset(log(duration))
                 , data = train, method = 'REML', family = nb)
    
    # predict
    pred.gam <- as.vector(predict(m.gam, validate))
    
    # inspect prediction vs actual
    data.frame(pred.gam, validate$species_obs)
plot(pred.gam ~ validate$species_obs)
    
    # calculate RMSE
    gam.rmse <- RMSE(f = pred.gam, o = validate$species_obs)
    
    
    # spaMM
    m.spamm <- fitme(species_obs ~ 
                            + temp
                            + rainfall
                            + week
                            + offset(log(duration))
                            + Matern(1|lat+lon)
                            , data=train, family=spaMM::negbin())
    
    # predict
    pred.spaMM <- as.vector(predict(m.spamm, validate))
    
    # inspect prediction vs actual
    data.frame(pred.spaMM, validate$species_obs)
plot(pred.spaMM ~ validate$species_obs)
    
    # calculate RMSE
    spaMM.rmse <- RMSE(f = pred.spaMM, o = validate$species_obs)
    
    # compare
    gam.rmse
    spaMM.rmse
    
    # diagnostic plots
    gratia::appraise(m.gam)
    simulationOutput <- DHARMa::simulateResiduals(m.spamm)
    plot(simulationOutput)

Data:

df <- structure(list(ID = c(398L, 425L, 311L, 66L, 295L, 316L, 2L, 
134L, 67L, 44L, 215L, 359L, 40L, 63L, 161L, 343L, 331L, 346L, 
415L, 326L, 349L, 78L, 228L, 431L, 123L, 406L, 420L, 272L, 419L, 
291L, 113L, 380L, 117L, 26L, 266L, 16L, 324L, 369L, 253L, 333L, 
409L, 265L, 309L, 160L, 109L, 363L, 169L, 105L, 147L, 184L, 204L, 
8L, 286L, 45L, 257L, 111L, 198L, 154L, 58L, 277L, 372L, 362L, 
410L, 385L, 175L, 61L, 304L, 34L, 102L, 149L, 301L, 255L, 407L, 
261L, 17L, 140L, 312L, 345L, 133L, 190L, 354L, 88L, 18L, 285L, 
15L, 314L, 207L, 397L, 336L, 239L, 163L, 315L, 86L, 402L, 387L, 
64L, 90L, 62L, 22L, 247L, 251L, 240L, 292L, 94L, 167L, 80L, 353L, 
394L, 75L, 323L, 427L, 322L, 244L, 356L, 214L, 104L, 373L, 367L, 
408L, 276L, 434L, 55L, 213L, 37L, 379L, 115L, 278L, 317L, 196L, 
5L, 327L, 243L, 318L, 211L, 237L, 186L, 335L, 51L, 32L, 106L, 
70L, 222L, 69L, 125L, 53L, 56L, 191L, 328L, 284L, 126L, 412L, 
1L, 185L, 82L, 194L, 334L, 170L, 360L, 400L, 437L, 224L, 281L, 
413L, 52L, 405L, 101L, 108L, 131L, 201L, 83L, 89L, 159L, 424L, 
216L, 249L, 65L, 283L, 174L, 260L, 57L, 47L, 435L, 297L, 432L, 
337L, 4L, 24L, 152L, 46L, 39L, 289L, 23L, 59L, 98L, 107L, 275L, 
258L, 221L, 296L, 81L, 294L, 195L, 176L, 245L, 230L, 389L, 54L, 
205L, 60L, 377L, 118L, 143L, 3L, 231L, 422L, 332L, 371L, 124L, 
274L, 384L, 130L, 302L, 202L, 13L, 310L, 421L, 110L, 138L, 173L, 
193L, 150L, 352L, 376L, 438L, 429L, 264L, 252L, 73L, 129L, 212L, 
279L, 341L, 430L, 43L, 411L, 181L, 232L, 338L, 114L, 401L), species_obs = c(16, 
5, 33, 3, 61, 7, 2, 4, 12, 72, 21, 25, 3, 31, 34, 59, 28, 381, 
34, 45, 149, 55, 34, 10, 3, 26, 2, 28, 2, 7, 44, 9, 14, 4, 60, 
4, 6, 8, 56, 118, 11, 80, 105, 119, 71, 0, 13, 34, 12, 48, 7, 
3, 133, 34, 8, 69, 127, 125, 7, 9, 50, 5, 9, 80, 11, 11, 51, 
13, 21, 67, 36, 153, 36, 12, 4, 31, 51, 75, 28, 22, 12, 5, 5, 
36, 16, 29, 6, 65, 10, 11, 2, 3, 11, 36, 10, 6, 30, 1, 19, 9, 
55, 9, 18, 16, 19, 18, 31, 388, 54, 5, 65, 39, 54, 5, 7, 14, 
98, 25, 115, 55, 15, 26, 22, 28, 17, 11, 62, 1, 87, 2, 19, 8, 
40, 2, 50, 21, 20, 6, 10, 41, 7, 56, 5, 6, 64, 16, 38, 1, 18, 
5, 8, 4, 48, 7, 66, 19, 7, 12, 21, 263, 22, 16, 14, 37, 39, 14, 
50, 8, 19, 20, 0, 61, 9, 72, 38, 1, 28, 5, 80, 103, 2, 27, 98, 
48, 11, 1, 10, 17, 29, 2, 146, 13, 12, 0, 3, 232, 12, 37, 51, 
29, 25, 38, 4, 42, 27, 18, 13, 7, 16, 15, 12, 35, 5, 14, 33, 
65, 5, 8, 25, 13, 2, 238, 4, 9, 38, 24, 32, 0, 17, 7, 7, 300, 
4, 430, 23, 93, 32, 37, 11, 3, 12, 26, 4, 7, 4, 30, 16, 28, 23, 
11), lat = c(51.451129, 51.502218, 51.489532, 51.495132, 51.511491, 
51.465341, 51.455139, 51.456529, 51.454777, 51.521703, 51.467293, 
51.487762, 51.475824, 51.503929, 51.498542, 51.505365, 51.496828, 
51.50229, 51.505993, 51.509689, 51.495543, 51.514119, 51.506038, 
51.482918, 51.448155, 51.472044, 51.44606, 51.495024, 51.523916, 
51.529513, 51.498967, 51.503607, 51.476284, 51.460305, 51.489773, 
51.516005, 51.460617, 51.492597, 51.505213, 51.454038, 51.502525, 
51.489721, 51.48483, 51.464065, 51.490967, 51.474593, 51.506125, 
51.468438, 51.486341, 51.502381, 51.465007, 51.523942, 51.501124, 
51.527381, 51.478232, 51.496083, 51.475137, 51.495459, 51.503625, 
51.461752, 51.529997, 51.476273, 51.503237, 51.496858, 51.505287, 
51.504638, 51.49083, 51.478764, 51.469812, 51.498601, 51.504588, 
51.504984, 51.472028, 51.503783, 51.507838, 51.476866, 51.489107, 
51.5039, 51.454272, 51.506429, 51.492034, 51.511654, 51.507251, 
51.477878, 51.511867, 51.488981, 51.45982, 51.499169, 51.455187, 
51.525597, 51.473402, 51.498695, 51.503315, 51.491199, 51.485027, 
51.475229, 51.50538, 51.504935, 51.501689, 51.496077, 51.505777, 
51.475398, 51.528991, 51.500417, 51.479416, 51.512185, 51.49177, 
51.504209, 51.480209, 51.46075, 51.490655, 51.477034, 51.500956, 
51.46164, 51.463871, 51.469704, 51.495696, 51.515154, 51.503549, 
51.469998, 51.483144, 51.495747, 51.480499, 51.487718, 51.504252, 
51.497831, 51.476197, 51.465557, 51.493491, 51.493815, 51.509747, 
51.451954, 51.480603, 51.456252, 51.518832, 51.501772, 51.489542, 
51.500988, 51.469842, 51.468268, 51.466462, 51.494986, 51.466056, 
51.478219, 51.49396, 51.502813, 51.506291, 51.447881, 51.477994, 
51.47869, 51.501041, 51.463646, 51.502, 51.526529, 51.511711, 
51.455286, 51.509251, 51.490036, 51.456044, 51.503862, 51.493901, 
51.457903, 51.505631, 51.509094, 51.497345, 51.468171, 51.491069, 
51.47711, 51.462577, 51.527362, 51.494216, 51.474142, 51.451633, 
51.471114, 51.4506, 51.461304, 51.473919, 51.50548, 51.502902, 
51.509742, 51.519995, 51.483867, 51.49833, 51.481017, 51.489147, 
51.448391, 51.459874, 51.503675, 51.527475, 51.475852, 51.502027, 
51.450253, 51.503561, 51.459049, 51.477071, 51.469962, 51.477865, 
51.48492, 51.51172, 51.510804, 51.520265, 51.497978, 51.461907, 
51.500177, 51.500548, 51.502364, 51.514178, 51.449365, 51.502786, 
51.517183, 51.475593, 51.47679, 51.453867, 51.500773, 51.521793, 
51.453182, 51.492361, 51.449356, 51.50472, 51.48996, 51.469197, 
51.49745, 51.474116, 51.514055, 51.48489, 51.521932, 51.484083, 
51.466515, 51.509177, 51.511645, 51.50507, 51.499093, 51.502571, 
51.50666, 51.494433, 51.495855, 51.502271, 51.46686, 51.469157, 
51.456329, 51.461621, 51.496366, 51.483239, 51.447783, 51.501109, 
51.499727, 51.516029, 51.496403, 51.499022, 51.48968), lon = c(17.32712, 
17.339485, 17.297313, 17.282313, 17.335249, 17.358213, 17.348407, 
17.36332, 17.317548, 17.360231, 17.308278, 17.295559, 17.382053, 
17.288016, 17.268009, 17.354437, 17.293116, 17.33772, 17.350742, 
17.29113, 17.319834, 17.346936, 17.329091, 17.290514, 17.342086, 
17.26711, 17.336913, 17.304416, 17.339454, 17.362003, 17.30583, 
17.384091, 17.293661, 17.286183, 17.320187, 17.308189, 17.356476, 
17.287544, 17.34188, 17.326814, 17.35231, 17.320066, 17.31377, 
17.368635, 17.311181, 17.278348, 17.290977, 17.303019, 17.305527, 
17.315307, 17.362876, 17.366789, 17.369126, 17.370542, 17.370688, 
17.325418, 17.359193, 17.322926, 17.314408, 17.366227, 17.364199, 
17.267791, 17.3518, 17.364918, 17.29854, 17.330414, 17.31632, 
17.29293, 17.303164, 17.310034, 17.326356, 17.331351, 17.267669, 
17.320875, 17.326238, 17.344633, 17.297602, 17.342287, 17.296472, 
17.325759, 17.316418, 17.293941, 17.325551, 17.375733, 17.341555, 
17.290418, 17.275607, 17.318046, 17.32659, 17.368091, 17.302578, 
17.385793, 17.366573, 17.313313, 17.375966, 17.340767, 17.293813, 
17.28353, 17.322853, 17.379632, 17.352715, 17.37553, 17.362152, 
17.2958, 17.274387, 17.347433, 17.31665, 17.335299, 17.301497, 
17.356474, 17.286939, 17.2657, 17.371728, 17.360524, 17.307898, 
17.303153, 17.301091, 17.345142, 17.391707, 17.369668, 17.312425, 
17.311435, 17.37221, 17.310367, 17.383809, 17.284236, 17.362692, 
17.358451, 17.285416, 17.389233, 17.290958, 17.319805, 17.334768, 
17.29626, 17.363543, 17.318716, 17.285766, 17.344752, 17.305798, 
17.303115, 17.310863, 17.314266, 17.310496, 17.282201, 17.301387, 
17.312488, 17.325604, 17.330627, 17.375799, 17.281866, 17.379533, 
17.345259, 17.31551, 17.356016, 17.30493, 17.325611, 17.291972, 
17.292673, 17.322426, 17.337843, 17.318787, 17.358639, 17.351146, 
17.345786, 17.2818, 17.298489, 17.311091, 17.379363, 17.355458, 
17.357466, 17.376965, 17.367449, 17.323354, 17.309375, 17.347163, 
17.356486, 17.369778, 17.298896, 17.320922, 17.332291, 17.350944, 
17.317929, 17.34012, 17.29532, 17.285625, 17.32001, 17.298458, 
17.317935, 17.370293, 17.382213, 17.374595, 17.29826, 17.31438, 
17.27979, 17.298206, 17.369555, 17.37049, 17.306157, 17.335395, 
17.3286, 17.363004, 17.272818, 17.283137, 17.37245, 17.288305, 
17.343277, 17.321669, 17.340615, 17.312236, 17.380145, 17.294091, 
17.345046, 17.336208, 17.28809, 17.350793, 17.326817, 17.287156, 
17.345781, 17.35423, 17.389642, 17.28138, 17.317021, 17.274507, 
17.340184, 17.3138, 17.351766, 17.302942, 17.357397, 17.297564, 
17.304735, 17.318904, 17.317477, 17.291807, 17.33538, 17.287692, 
17.304503, 17.35348, 17.302022, 17.281458, 17.296259, 17.366458, 
17.368954, 17.292115, 17.292487, 17.379349, 17.312119, 17.301971, 
17.362325, 17.305982, 17.312158), duration = c(70.1355555555555, 
50.7836111111112, 121.600833333333, 84.4341666666667, 66.2302777777778, 
63.9608333333333, 75.6888888888888, 67.15, 59.9852777777778, 
72.9563888888888, 72.4122222222222, 61.8591666666667, 72.9563888888888, 
96.7561111111112, 75.4586111111112, 82.2433333333333, 54.6652777777778, 
63.5977777777778, 50.7836111111112, 73.3108333333333, 63.5977777777778, 
70.2347222222222, 40.7666666666667, 41.1544444444445, 67.15, 
51.0241666666667, 50.7836111111112, 47.5847222222222, 59.3911111111112, 
76.3677777777778, 67.5336111111112, 61.5280555555555, 67.5336111111112, 
87.6202777777778, 69.0416666666667, 75.2983333333333, 63.9608333333333, 
61.5280555555555, 59.3541666666667, 54.6652777777778, 50.7836111111112, 
69.0416666666667, 66.2302777777778, 64.8672222222222, 69.4605555555555, 
43.9444444444445, 75.4586111111112, 81.2630555555555, 78.1188888888888, 
75.0188888888888, 83.0036111111112, 75.2983333333333, 66.6222222222222, 
48.3775, 69.455, 93.1305555555555, 83.0036111111112, 78.1188888888888, 
84.8886111111112, 37.7319444444445, 52.5955555555555, 52.8769444444445, 
50.7836111111112, 61.2030555555555, 75.4586111111112, 84.8886111111112, 
66.2302777777778, 87.6202777777778, 69.4605555555555, 78.1188888888888, 
66.2302777777778, 59.3541666666667, 51.0241666666667, 69.0416666666667, 
75.2983333333333, 78.5658333333333, 121.600833333333, 63.5977777777778, 
67.15, 75.0188888888888, 45.1677777777778, 69.8472222222222, 
75.2983333333333, 56.9347222222222, 75.2983333333333, 83.1941666666667, 
72.4122222222222, 61.2030555555555, 63.9608333333333, 69.8705555555555, 
75.4586111111112, 63.9608333333333, 93.6469444444445, 61.2030555555555, 
70.1355555555555, 109.143055555556, 69.8472222222222, 96.7561111111112, 
75.2983333333333, 69.455, 59.3541666666667, 69.455, 76.3677777777778, 
81.7147222222222, 75.4586111111112, 70.2347222222222, 63.5977777777778, 
61.2030555555555, 84.4341666666667, 63.9608333333333, 67.7219444444445, 
73.3108333333333, 69.455, 52.8769444444445, 72.4122222222222, 
81.2630555555555, 61.5280555555555, 52.5955555555555, 59.3911111111112, 
96.0336111111112, 41.1544444444445, 72.5666666666667, 83.0036111111112, 
87.6202777777778, 61.5280555555555, 67.5336111111112, 37.7319444444445, 
63.9608333333333, 62.2511111111112, 75.6888888888888, 63.9608333333333, 
69.455, 63.9608333333333, 72.4122222222222, 59.7094444444445, 
75.0188888888888, 73.3108333333333, 84.8886111111112, 87.6202777777778, 
81.2630555555555, 84.4341666666667, 82.5130555555555, 84.4341666666667, 
55.7986111111112, 72.5666666666667, 84.8886111111112, 75.0188888888888, 
63.9608333333333, 66.6222222222222, 67.15, 59.3911111111112, 
75.6888888888888, 75.0188888888888, 81.7147222222222, 62.2511111111112, 
63.9608333333333, 75.4586111111112, 61.8591666666667, 43.4841666666667, 
41.1544444444445, 82.5130555555555, 66.6222222222222, 50.7836111111112, 
84.8886111111112, 51.0241666666667, 81.2630555555555, 93.1305555555555, 
78.5658333333333, 83.0036111111112, 81.7147222222222, 93.6469444444445, 
42.9927777777778, 50.7836111111112, 72.4122222222222, 69.455, 
109.143055555556, 66.6222222222222, 53.5841666666667, 69.0416666666667, 
84.8886111111112, 84.8886111111112, 41.1544444444445, 75.9177777777778, 
41.1544444444445, 73.3108333333333, 75.6888888888888, 87.6202777777778, 
66.7675, 72.9563888888888, 85.3433333333333, 66.6222222222222, 
87.6202777777778, 84.8886111111112, 81.2630555555555, 81.2630555555555, 
105.955, 69.455, 93.1044444444445, 66.2302777777778, 70.2347222222222, 
76.3677777777778, 62.2511111111112, 75.4586111111112, 69.455, 
69.8705555555555, 61.2030555555555, 84.8886111111112, 83.0036111111112, 
84.8886111111112, 61.5280555555555, 67.5336111111112, 67.15, 
75.6888888888888, 59.7094444444445, 59.3911111111112, 63.9608333333333, 
61.5280555555555, 55.7986111111112, 57.2722222222222, 70.1355555555555, 
67.15, 66.2302777777778, 72.4122222222222, 75.2983333333333, 
66.2302777777778, 59.3911111111112, 81.2630555555555, 78.5658333333333, 
75.4586111111112, 72.8425, 78.1188888888888, 63.5977777777778, 
61.5280555555555, 41.1544444444445, 67.7219444444445, 69.0416666666667, 
59.3541666666667, 84.4341666666667, 67.15, 72.4122222222222, 
66.6222222222222, 72.8933333333333, 41.1544444444445, 60.6344444444445, 
59.3911111111112, 75.0188888888888, 69.8705555555555, 72.8933333333333, 
67.5336111111112, 61.2030555555555), week = c(3, 2, 4, 11, 5, 
4, 12, 9, 11, 11, 7, 3, 11, 11, 8, 4, 4, 4, 2, 4, 4, 10, 7, 1, 
9, 2, 2, 5, 2, 5, 9, 3, 9, 12, 6, 12, 4, 3, 6, 4, 2, 6, 5, 8, 
10, 3, 8, 10, 9, 8, 7, 12, 5, 11, 6, 10, 7, 9, 11, 5, 3, 3, 2, 
3, 8, 11, 5, 12, 10, 9, 5, 6, 2, 6, 12, 9, 4, 4, 9, 8, 4, 10, 
12, 5, 12, 4, 7, 3, 4, 6, 8, 4, 10, 3, 3, 11, 10, 11, 12, 6, 
6, 6, 5, 10, 8, 10, 4, 3, 11, 4, 2, 4, 6, 3, 7, 10, 3, 3, 2, 
5, 1, 11, 7, 12, 3, 9, 5, 4, 7, 12, 4, 6, 4, 7, 6, 8, 4, 11, 
12, 10, 11, 7, 11, 9, 11, 11, 8, 4, 5, 9, 2, 12, 8, 10, 7, 4, 
8, 3, 3, 1, 7, 5, 2, 11, 2, 10, 10, 9, 7, 10, 10, 8, 2, 7, 6, 
11, 5, 8, 6, 11, 11, 1, 5, 1, 4, 12, 12, 9, 11, 11, 5, 12, 11, 
10, 10, 5, 6, 7, 5, 10, 5, 7, 8, 6, 6, 3, 11, 7, 11, 3, 9, 9, 
12, 6, 2, 4, 3, 9, 5, 3, 9, 5, 7, 12, 5, 2, 10, 9, 8, 7, 9, 4, 
3, 1, 2, 6, 6, 11, 9, 7, 5, 4, 1, 11, 2, 8, 6, 4, 9, 3), temp = structure(c(0.379862311883938, 
-0.0590348398838672, 0.93156013349855, -0.388207703709687, 1.4277454730557, 
0.295383582949186, -2.08944689712732, -0.662005693247078, -0.0939005014728701, 
-0.477422778952212, 0.666991289675932, 0.516248576335129, -0.563561472289814, 
-0.806937462989702, -0.485113733714489, 0.177159809775178, 0.116934205077238, 
-0.0220549197503717, 0.113242546791336, 0.307397432366173, -0.0220549197503717, 
-0.23131222655907, 1.02877380169386, -0.305145392277009, -0.108256950362482, 
-0.169784588460759, -0.00981272940521774, 1.75808140528556, 0.699293299677533, 
1.37524276822956, -0.98194941135816, 0.354738526327126, -0.23131222655907, 
-2.24018961046812, 1.35610083637674, -1.74489212377691, 0.295383582949186, 
0.473179229666329, 1.60221138876989, 0.116934205077238, 0.0886314915520457, 
1.56068023305355, 1.4277454730557, -0.194395643700083, -0.243617754178698, 
-0.00489051835735971, -0.635856447055291, -0.58201976371929, 
0.365505862994327, -1.1526886070809, 0.389433277810312, -2.23711322856321, 
1.4802770246745, 0.0640204363127213, 0.946942043023137, -0.165682745920894, 
0.494713903000729, 0.182461139651923, -0.614321773720891, 1.31691315981737, 
-0.0344237846445428, 0.23193401403373, 0.37165862680414, 0.0640204363127213, 
-1.0880845870777, -0.926574537069696, 1.4277454730557, -1.68028810377371, 
-0.883505190400896, -0.215930317034484, 1.19459078636874, 1.29457319827847, 
-0.132868005601806, 1.17305611303434, -2.53244589143498, -1.0019458937401, 
1.09153199255413, -0.0220549197503717, 0.0640204363127213, -1.1203865970793, 
-0.00289187547271968, -1.26497654661027, -2.75394538858882, 1.66373902686816, 
-2.45861272571704, 0.54639711900329, 0.936174706355937, 0.430109882997528, 
0.295383582949186, 0.979244053024738, -0.678925793724092, 0.295383582949186, 
-0.127398882215297, 0.623921943007131, 0.216638434350237, -0.599247502386811, 
-0.93272730087951, -0.854792292621707, -2.24941875618286, 1.03308073636074, 
1.29457319827847, 0.914640033021536, 1.51091629793462, -0.786599160396094, 
-0.754297150394493, -0.378978557994949, -0.0220549197503717, 
0.294936035774541, -0.129791623696882, 0.295383582949186, 0.0257365726071244, 
0.307397432366173, 0.850036013018335, 0.23193401403373, 0.925407369688737, 
-1.0234805670745, 0.333203852992726, 0.445491792522114, 0.333203852992726, 
1.54403980365877, -0.194395643700083, -0.280534337037685, 0.46600100522154, 
-2.05714488712572, 0.272911938046161, -0.169784588460759, 1.31691315981737, 
0.295383582949186, 0.740824455393871, -1.69567001329828, 0.295383582949186, 
1.45300686638155, 0.295383582949186, 0.591619933005531, 1.1961289773212, 
-1.1957579537497, 0.307397432366173, -0.754297150394493, -1.88486750045051, 
-0.754297150394493, -0.205162980367284, 0.753129983013534, -0.0436529303592806, 
-0.811722945952906, -0.600478055148767, -0.926574537069696, -1.2065252904169, 
0.295383582949186, 1.4802770246745, -0.0836458951231574, -0.0436529303592806, 
-1.7202810685376, -0.969643883738497, -0.398975040376887, 0.740824455393871, 
0.295383582949186, -0.851203180399295, 0.343741969047418, 0.896694471909548, 
-0.489728306571875, 0.944549301541552, 1.4802770246745, 0.0886314915520457, 
-0.797366497063294, -0.0836458951231574, -1.3249659937561, -0.280534337037685, 
-0.711227803725692, 0.79141384671913, -0.463579060380088, 0.433186264902452, 
-1.57261473710171, 0.00249279821444417, 0.731595309679133, 1.14075410303274, 
-0.831821974398335, 1.4802770246745, -0.481524621492077, 1.67912093639275, 
-0.948109210404097, -0.818901170397695, -0.120562477982144, 1.5207151440021, 
-0.0344237846445428, 0.307397432366173, -2.05253031426834, -2.26172428380252, 
-0.0836458951231574, -0.723533331345355, -1.0988519237449, 1.4802770246745, 
-2.01407554045692, -0.754297150394493, -0.495881070381689, -0.334371020373686, 
1.34892261193196, 1.06538274636234, 0.590423562264738, 1.15152143969994, 
-0.403589613234273, 1.51091629793462, 0.986935007787013, -0.463579060380088, 
0.796199329682334, 1.01154606302634, -0.162093633698483, -0.700460467058492, 
0.772271914866349, -1.0127132304073, 0.440877219664728, -0.735838858965017, 
-0.194395643700083, -1.8064197618752, 1.07307370112461, -0.495881070381689, 
0.295383582949186, 0.272911938046161, -0.811722945952906, 1.47109813847757, 
0.216638434350237, -0.526644889430827, 1.38840284637834, 0.634689279674331, 
-2.5570569466743, 1.4277454730557, 0.0317184263111207, -0.0544202670264808, 
-0.0974896136952816, -0.829668507064895, 0.796199329682334, -0.215930317034484, 
-0.0220549197503717, 0.333203852992726, -0.120562477982144, -0.098685984436074, 
1.57144756972075, 1.22074003256053, -0.302069010372085, -0.46511725133255, 
0.623921943007131, 1.4802770246745, 0.152009985561075, -0.342061975135962, 
0.0353075385335322, 0.0424857629783209, -0.345138357040886, 1.54991289638635, 
0.152009985561075, -0.920421773259883, 0.322436516325526), .Dim = c(250L, 
1L)), rainfall = structure(c(-0.42221818637359, 0.32243062926338, 
-0.541893888886675, -0.724731767726109, -0.352407359907624, -0.77127231870342, 
-0.26265058302281, -0.741353393075149, -0.794542594192075, -0.661569591399759, 
2.16078239286715, -0.375677635396279, -0.661569591399759, -0.691119147575829, 
0.997268618434384, -0.775926373801151, -0.794542594192075, -0.794542594192075, 
0.32243062926338, -0.773857904868826, -0.794542594192075, -0.688164191958222, 
1.29047408959144, -0.289245183581274, -0.741353393075149, 0.32243062926338, 
0.32243062926338, -0.236055982464348, 0.182808976331448, -0.236055982464348, 
-0.741353393075149, -0.375677635396279, -0.741353393075149, -0.42221818637359, 
0.997268618434384, -0.448812786932054, -0.77127231870342, -0.375677635396279, 
0.774538838757258, -0.794542594192075, 0.32243062926338, 0.997268618434384, 
-0.352407359907624, 1.01389024378342, -0.688164191958222, -0.236055982464348, 
0.997268618434384, -0.701461492237454, -0.748002043214764, 1.04380916941169, 
3.23897182384151, -0.448812786932054, -0.352407359907624, -0.794542594192075, 
0.578403659638588, -0.711803836899078, 3.23897182384151, -0.748002043214764, 
-0.678191216748798, -0.757310153410226, -0.315839784139738, -0.315839784139738, 
0.32243062926338, -0.375677635396279, 0.997268618434384, -0.678191216748798, 
-0.352407359907624, -0.42221818637359, -0.688164191958222, -0.748002043214764, 
-0.352407359907624, 0.774538838757258, 0.32243062926338, 0.997268618434384, 
-0.448812786932054, -0.748002043214764, -0.541893888886675, -0.794542594192075, 
-0.741353393075149, 1.04380916941169, -0.794542594192075, -0.688164191958222, 
-0.448812786932054, -0.289245183581274, -0.448812786932054, -0.775926373801151, 
2.16078239286715, -0.375677635396279, -0.77127231870342, 0.857646965502452, 
0.997268618434384, -0.77127231870342, -0.711803836899078, -0.375677635396279, 
-0.42221818637359, -0.701461492237454, -0.688164191958222, -0.691119147575829, 
-0.448812786932054, 0.578403659638588, 0.774538838757258, 0.578403659638588, 
-0.236055982464348, -0.701461492237454, 0.997268618434384, -0.688164191958222, 
-0.794542594192075, -0.375677635396279, -0.724731767726109, -0.77127231870342, 
0.0742143573843895, -0.773857904868826, 0.578403659638588, -0.315839784139738, 
2.16078239286715, -0.701461492237454, -0.375677635396279, -0.315839784139738, 
0.182808976331448, 0.0178015683209832, -0.289245183581274, -0.714758792516685, 
3.23897182384151, -0.42221818637359, -0.375677635396279, -0.741353393075149, 
-0.757310153410226, -0.77127231870342, 2.58297167673275, -0.26265058302281, 
-0.77127231870342, 0.578403659638588, -0.77127231870342, 2.16078239286715, 
0.774538838757258, 1.04380916941169, -0.773857904868826, -0.678191216748798, 
-0.42221818637359, -0.701461492237454, -0.724731767726109, 2.10131391106281, 
-0.724731767726109, -0.732488526222328, -0.714758792516685, -0.678191216748798, 
1.04380916941169, -0.77127231870342, -0.352407359907624, -0.741353393075149, 
0.182808976331448, -0.26265058302281, 1.04380916941169, -0.701461492237454, 
2.58297167673275, -0.77127231870342, 0.997268618434384, -0.375677635396279, 
-0.236055982464348, -0.289245183581274, 2.10131391106281, -0.352407359907624, 
0.32243062926338, -0.678191216748798, 0.32243062926338, -0.701461492237454, 
-0.711803836899078, -0.748002043214764, 3.23897182384151, -0.701461492237454, 
-0.711803836899078, 1.73726337897362, 0.32243062926338, 2.16078239286715, 
0.578403659638588, -0.701461492237454, -0.352407359907624, 1.59453902264321, 
0.997268618434384, -0.678191216748798, -0.678191216748798, -0.289245183581274, 
-0.401533497050341, -0.289245183581274, -0.773857904868826, -0.26265058302281, 
-0.42221818637359, -0.741353393075149, -0.661569591399759, -0.678191216748798, 
-0.352407359907624, -0.42221818637359, -0.678191216748798, -0.701461492237454, 
-0.701461492237454, 0.213836010316318, 0.578403659638588, 3.07763124712017, 
-0.352407359907624, -0.688164191958222, -0.236055982464348, 2.58297167673275, 
0.997268618434384, 0.578403659638588, 0.857646965502452, -0.375677635396279, 
-0.678191216748798, 3.23897182384151, -0.678191216748798, -0.375677635396279, 
-0.741353393075149, -0.741353393075149, -0.26265058302281, 0.774538838757258, 
0.182808976331448, -0.77127231870342, -0.375677635396279, -0.732488526222328, 
-0.315839784139738, -0.42221818637359, -0.741353393075149, -0.352407359907624, 
2.16078239286715, -0.448812786932054, -0.352407359907624, 0.182808976331448, 
-0.701461492237454, -0.748002043214764, 0.997268618434384, 3.74316112609571, 
-0.748002043214764, -0.794542594192075, -0.375677635396279, -0.289245183581274, 
0.0742143573843895, 0.997268618434384, 0.774538838757258, -0.724731767726109, 
-0.741353393075149, 2.16078239286715, -0.352407359907624, -0.773857904868826, 
-0.289245183581274, -0.701461492237454, 0.182808976331448, 1.04380916941169, 
0.857646965502452, -0.773857904868826, -0.741353393075149, -0.375677635396279
), .Dim = c(250L, 1L))), row.names = c(427L, 455L, 339L, 74L, 
321L, 344L, 2L, 146L, 75L, 51L, 235L, 388L, 47L, 71L, 173L, 372L, 
359L, 375L, 445L, 354L, 378L, 87L, 250L, 462L, 134L, 436L, 450L, 
298L, 449L, 317L, 123L, 409L, 127L, 31L, 292L, 17L, 352L, 398L, 
279L, 361L, 439L, 291L, 337L, 172L, 119L, 392L, 183L, 115L, 159L, 
201L, 222L, 8L, 312L, 52L, 283L, 121L, 216L, 166L, 65L, 303L, 
401L, 391L, 440L, 414L, 190L, 69L, 332L, 40L, 112L, 161L, 328L, 
281L, 437L, 287L, 18L, 152L, 340L, 374L, 145L, 207L, 383L, 98L, 
19L, 311L, 16L, 342L, 225L, 426L, 364L, 263L, 175L, 343L, 96L, 
432L, 416L, 72L, 100L, 70L, 23L, 273L, 277L, 265L, 318L, 104L, 
180L, 89L, 382L, 423L, 83L, 351L, 457L, 350L, 269L, 385L, 232L, 
114L, 402L, 396L, 438L, 302L, 465L, 62L, 231L, 44L, 408L, 125L, 
304L, 345L, 214L, 5L, 355L, 268L, 346L, 229L, 261L, 203L, 363L, 
58L, 38L, 116L, 78L, 243L, 77L, 137L, 60L, 63L, 208L, 356L, 310L, 
138L, 442L, 1L, 202L, 92L, 211L, 362L, 184L, 389L, 430L, 468L, 
245L, 307L, 443L, 59L, 435L, 111L, 118L, 143L, 219L, 93L, 99L, 
171L, 454L, 237L, 275L, 73L, 309L, 189L, 286L, 64L, 54L, 466L, 
323L, 463L, 365L, 4L, 29L, 164L, 53L, 46L, 315L, 26L, 66L, 108L, 
117L, 301L, 284L, 242L, 322L, 90L, 320L, 213L, 191L, 270L, 253L, 
418L, 61L, 223L, 68L, 406L, 128L, 155L, 3L, 254L, 452L, 360L, 
400L, 135L, 300L, 413L, 142L, 330L, 220L, 14L, 338L, 451L, 120L, 
150L, 188L, 210L, 162L, 381L, 405L, 469L, 460L, 290L, 278L, 81L, 
141L, 230L, 305L, 369L, 461L, 50L, 441L, 197L, 255L, 366L, 124L, 
431L), class = "data.frame")
$\endgroup$
4
  • 1
    $\begingroup$ You're not comparing like with like; you have smooth of week in one model but linear in the other, a smooth interaction between space and week in one and no interaction at all in the other, you're assuming the spatial effect is smooth in one and likely rough in the other. Try fitting the same model in mgcv (as you did in the spaMM one) and crank k as high as it can go. Did you miss a bs = 'ds' on the first spatial smooth? I suspect the Matern gets a short estimated length scale & hence is very rough, esp as you have simple fixed effs compared to the GAM, whereas... $\endgroup$ Commented Jan 29, 2021 at 15:43
  • 1
    $\begingroup$ In the GAM you're assuming space is essentially smooth (and flat away from the data given the Duchon spline settings). $\endgroup$ Commented Jan 29, 2021 at 15:44
  • 1
    $\begingroup$ Also, I suspect you're comparing results from the GAM on the link scale with the response scale in spaMM. The default for predict.gam() is type = "link", but for models fitted by fitme(), the default is type = "response". Regardless, your comparison of the GAM predictions to the observed data makes no sense as you are comparing observed counts with link-scale (i.e. log-scale) predictions from the model. $\endgroup$ Commented Jan 29, 2021 at 15:50
  • 1
    $\begingroup$ Thanks @GavinSimpson. I'm aware that the models weren't exactly the same (I haven't worked out how to specify the interaction with week and location in the spaMM model). The issue was the prediction scale: type = "response" has fixed it - I hadn't spotted that the default in gam was link. If you want to put that as an answer I'll accept it. Thank again! $\endgroup$
    – Thomas
    Commented Feb 1, 2021 at 10:20

1 Answer 1

1
$\begingroup$

This was a simple fix - as pointed out by @GavinSimpson, the issue was the prediction scale: changing the GAM prediction to type = "response" has fixed it. Both approaches now perform similarly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.