Random Forest vs. General Additive Model for Predicting

Question

I built two models in R and python, a General Additive Model and a Random Forest model. Both models were built on the same dataset:

   Albedo Year_Since_Burn Summer_SRAD Winter_SRAD
1  397.00               1    17801.70     6589.56
2  289.60               2    18027.20     6633.96
3  615.29               3    17397.10     6952.69
4  258.12               4    17793.63     6627.62
5  139.32               5    17853.00     6675.00
6  463.81               6    17853.00     6675.00
7  532.47               7    17853.00     6675.00
8  300.09               8    17648.00     6890.00
9  118.00               9    17786.13     6724.67
10 238.18              10    18050.13     6916.46
11 439.11              11    18057.20     6893.08
12 366.00              12    17823.00     6618.12
13 441.25              13    17809.50     6673.79
14 450.31              14    17654.40     6849.19
15 275.43              15    17592.80     7202.88
16 147.11              16    17830.20     6672.88
17 285.68              17    18065.13     6897.58
18 309.61              18    17665.80     7036.62
19 264.95              19    18053.47     6867.17
20 125.18              20    17834.40     6661.19
21 289.50              21    17824.00     6684.50
22 293.61              22    17826.90     6681.83
23 368.95              23    17634.55     6914.06
24 563.11              24    17434.23     7043.04
25 434.41              25    17527.60     7070.38
26 199.78              26    17955.40     6704.00
27 153.37              27    17872.70     6637.00
28 287.29              28    17843.20     6659.67
29 173.52              29    17822.93     6616.75
30 239.28              30    17884.00     6580.56
31 292.91              31    17884.00     6580.56
32 323.00              32    18078.70     6758.50
33 282.00              33    18078.70     6758.50
34 237.50              34    17779.10     7303.38
35 225.00              35    17822.80     6617.42
36 237.55              36    17822.80     6617.42
37 247.11              37    17918.50     6695.71
38 336.48              38    17918.50     6695.71
39 290.00              39    17918.50     6695.71
40 248.42              40    17822.80     6617.42
41 304.74              41    17918.50     6695.71
42 311.52              42    17918.50     6695.71
43 281.39              43    17918.50     6695.71
44 234.68              44    17918.50     6695.71
45 297.58              45    17918.50     6695.71
46 265.52              46    17918.50     6695.71
47 186.29              47    17918.50     6695.71
48 291.16              48    17918.50     6695.71
49 185.17              49    17918.50     6695.71
50 288.94              50    17918.50     6695.71
51 269.64              51    17918.50     6695.71
52 255.00              52    17918.50     6695.71
.................................................
70 260.14              70    17918.50     6695.71

I predicted Albedo based on Year_Since_burn, Summer_SRAD and Winter_SRAD for both. I then used these models to predict on two new dataset. To create the new dataset I picked two different Summer_SRAD and Winter_SRAD variables and then duplicated them 70 times, while adding a column for Year_Since_Burn.

An example of this would be:

   Year_Since_Burn  Summer_SRAD  Winter_SRAD
   1                17801.70     6589.56
   2                17801.70     6589.56
   3                17801.70     6589.56
   4                17801.70     6589.56
   5                17801.70     6589.56
   6                17801.70     6589.56
   .....................................        
   70               17801.70     6589.56

On these two new datasets I then predicted Albedo, and plotted predicted Albedo vs. Year Since Burn.

For the General Additive Model the predicted curves are identical, but for the Random Forest they are different. For the Random Forest the curves should be different right? Because with different inputs for the independent variables different routes in the tree will be taken and thus a different output. What I don't understand is why for the General Additive Model the predicted curves have the same shapes, but shift up or down on the y-axis.

An example of two General Additive Curves are:

For the Random Forest the curves vary:

For more details on how I created the GAM:

https://stackoverflow.com/questions/46221975/gam-predictions-in-r-have-same-curve-shape

The Random Forest was done in the h2o package in python in a similar manner.

Answer 1

The GAM curves aren't identical. They have the same shape because the only thing varying in the prediction is Years_Since_Burn; you've predicted from an additive model varying a single covariate whilst keeping the other covariates fixed. Hence what you are seeing is the effect of Years_Since_Burn shifted up or down as determined by the additive effects of the static values of the other two covariates.

Random forest models include potentially complex interactions between covariates. It is not surprising therefore that the two curves are different when given different, albeit static, values of the other two covariates.

Your GAM is strictly additive and includes no interactions (as you've fitted it). Hence by definition the effect of Years_Since_Burn is unchanged by the values taken by the other covariates. Therefore, the only change in the predicted values is a mean shift due to adding on the fixed effect of the other two covariates.

If you want the GAM to include interactions, you will need to fit smoothers of more than a single variable. For example for two-way interactions:

model <- gam(Albedo ~ te(Year_Since_Burn, Summer_SRAD) +
               te(Summer_SRAD, Winter_SRAD),
             data=df)

or even (for a three-way interaction)

model <- gam(Albedo ~ te(Year_Since_Burn, Summer_SRAD, Winter_SRAD),
             data=df)

but I doubt you have enough data for that final model given the curse of dimensionality and the problem of smoothing in higher dimensions.

You might also need to be careful about how you go about setting up the splines in the 2-way-interaction GAM because, as written above, Summer_SRDA exists in two tensor product smooths.