# How to evaluate a Gamma GLM fit and appropriateness for the data?

I am trying to predict the time (in seconds) a process would take based on the computer's disk throughput (measured by dd and reported in MiB/s) and the amount of user data that needs to be processed. I have collected data with several levels of user data (64, 124, 256, 512, 1024, and 2048 MiB) on several Amazon EC2 instance/computer types. I have around 2500 observations.

The time (ec2$elapsed_time) and the throughput (ec2$dd_128k) look like this:

Because I do not control the throughput of the disks that I get assigned per instance, there are some gaps in the histogram. So I decided to use binning for the throughput data. I grouped it in 8 categories (bin #5 has no observations):

• Bin 1: <= 15
• Bin 2: 15.01 - 35
• Bin 3: 35.01 - 55
• Bin 4: 55.01 - 75
• Bin 5: 75.01 - 95
• Bin 6: 95.01 - 115
• Bin 7: 115.01 - 135
• Bin 8: 135+

To visualize the trend, I plotted the mean for each bin and grouped it by the amount of user data:

At first, I attempted to use a linear regression but the model could not explain the variation in the data properly and given that the data did not seem linear, I figured it was not appropriate. I was looking into using a GLM with a gamma distribution.

Using the script below, I attempted to fit a Gamma GLM to the data. Where dd_128k_bin is a categorical value (1-8) and mem_factor is also categorical (64, 128, 256, 512, 2048).

model <- glm(elapsed_time ~ dd_128k_bin * mem_factor, data=ec2, family = "Gamma")


This is what I get in the summary:

Call:
glm(formula = elapsed_time ~ dd_128k_bin * mem_factor, family = "Gamma",
data = ec2)

Deviance Residuals:
Min        1Q    Median        3Q       Max
-1.84829  -0.22591  -0.00416   0.10916   1.63584

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)                   6.4516     0.5198  12.413  < 2e-16 ***
dd_128k_bin2                 -2.1475     0.5747  -3.737 0.000191 ***
dd_128k_bin3                 -1.6414     0.5659  -2.901 0.003757 **
dd_128k_bin4                 -0.5071     0.5887  -0.861 0.389128
dd_128k_bin6                 -2.5555     0.5652  -4.522 6.42e-06 ***
dd_128k_bin7                 -1.6285     0.5879  -2.770 0.005648 **
dd_128k_bin8                 -1.7198     0.6446  -2.668 0.007678 **
mem_factor128                -1.4433     0.6580  -2.193 0.028366 *
mem_factor256                -2.9428     0.5916  -4.974 7.01e-07 ***
mem_factor512                -6.0697     0.5207 -11.658  < 2e-16 ***
mem_factor1024               -6.4028     0.5198 -12.319  < 2e-16 ***
mem_factor2048               -6.4349     0.5198 -12.381  < 2e-16 ***
dd_128k_bin2:mem_factor128    0.7405     0.7315   1.012 0.311487
dd_128k_bin3:mem_factor128    0.5700     0.7187   0.793 0.427787
dd_128k_bin4:mem_factor128    0.2581     0.7473   0.345 0.729791
dd_128k_bin6:mem_factor128    0.8439     0.7194   1.173 0.240872
dd_128k_bin7:mem_factor128    0.6524     0.7491   0.871 0.383924
dd_128k_bin8:mem_factor128    0.7655     0.8276   0.925 0.355103
dd_128k_bin2:mem_factor256    1.4049     0.6595   2.130 0.033257 *
dd_128k_bin3:mem_factor256    0.9708     0.6462   1.502 0.133122
dd_128k_bin4:mem_factor256    0.3132     0.6710   0.467 0.640676
dd_128k_bin6:mem_factor256    1.4496     0.6466   2.242 0.025047 *
dd_128k_bin7:mem_factor256    0.9485     0.6719   1.412 0.158179
dd_128k_bin8:mem_factor256    0.9939     0.7387   1.346 0.178572
dd_128k_bin2:mem_factor512    3.1667     0.5810   5.450 5.52e-08 ***
dd_128k_bin3:mem_factor512    2.3968     0.5692   4.211 2.63e-05 ***
dd_128k_bin4:mem_factor512    2.0393     0.5962   3.420 0.000635 ***
dd_128k_bin6:mem_factor512    3.7534     0.5731   6.549 7.01e-11 ***
dd_128k_bin7:mem_factor512    3.0910     0.5980   5.169 2.54e-07 ***
dd_128k_bin8:mem_factor512    3.2402     0.6632   4.885 1.10e-06 ***
dd_128k_bin2:mem_factor1024   2.1949     0.5747   3.819 0.000137 ***
dd_128k_bin3:mem_factor1024   1.7589     0.5659   3.108 0.001906 **
dd_128k_bin4:mem_factor1024   0.7333     0.5889   1.245 0.213159
dd_128k_bin6:mem_factor1024   3.4384     0.5677   6.057 1.60e-09 ***
dd_128k_bin7:mem_factor1024   2.6521     0.5911   4.487 7.56e-06 ***
dd_128k_bin8:mem_factor1024   2.7587     0.6505   4.241 2.31e-05 ***
dd_128k_bin2:mem_factor2048   2.1517     0.5747   3.744 0.000185 ***
dd_128k_bin3:mem_factor2048   1.6706     0.5659   2.952 0.003185 **
dd_128k_bin4:mem_factor2048   0.5531     0.5887   0.939 0.347606
dd_128k_bin6:mem_factor2048   3.0518     0.5659   5.393 7.60e-08 ***
dd_128k_bin7:mem_factor2048   2.1888     0.5888   3.717 0.000206 ***
dd_128k_bin8:mem_factor2048   2.2907     0.6463   3.544 0.000401 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for Gamma family taken to be 0.1947056)

Null deviance: 8942.38  on 2519  degrees of freedom
Residual deviance:  661.38  on 2478  degrees of freedom
AIC: 2578.1

Number of Fisher Scoring iterations: 6


And these are the plots associated with the model:

Using the RsqGLM() function, I obtained an R^2 of 0.95 with the CoxSnell method.

I am concerned about the assumptions that need to be satisfied. For linear models, you are usually concerned with the Normal Q-Q plot following the drawn line closely. In this case, since the data is not normal, it does not. So I am wondering if this model is appropriate even when it's R^2 value seems appropriate. Is there anything else that I should consider to validate whether this model is appropriate?

Edit: I hadn't considered that binning could introduce bias. I checked the model without binning and these are the results:

model <- glm(elapsed_time ~ dd_128k * mem_factor, data=ec2, family = "Gamma")


R^2 = 0.93

glm(formula = elapsed_time ~ dd_128k * mem_factor, family = "Gamma",
data = ec2)

Deviance Residuals:
Min        1Q    Median        3Q       Max
-1.60095  -0.42446   0.01102   0.16540   1.70529

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)             5.304060   0.299458  17.712  < 2e-16 ***
dd_128k                -0.006009   0.003406  -1.764 0.077843 .
mem_factor128          -1.032953   0.387206  -2.668 0.007686 **
mem_factor256          -2.155936   0.348180  -6.192 6.92e-10 ***
mem_factor512          -4.866027   0.305453 -15.931  < 2e-16 ***
mem_factor1024         -5.361043   0.299505 -17.900  < 2e-16 ***
mem_factor2048         -5.320488   0.299461 -17.767  < 2e-16 ***
dd_128k:mem_factor128   0.002454   0.004428   0.554 0.579522
dd_128k:mem_factor256   0.002412   0.003960   0.609 0.542493
dd_128k:mem_factor512   0.019934   0.003572   5.580 2.66e-08 ***
dd_128k:mem_factor1024  0.012081   0.003413   3.540 0.000408 ***
dd_128k:mem_factor2048  0.007703   0.003407   2.261 0.023833 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for Gamma family taken to be 0.3468551)

Null deviance: 8942.38  on 2519  degrees of freedom
Residual deviance:  875.08  on 2508  degrees of freedom
AIC: 3259

Number of Fisher Scoring iterations: 6


• I don't understand the value of doing the binning; few things will be worth what it costs you in bias etc. Commented Sep 10, 2021 at 6:53
• @Glen_b thanks for your suggestion! I made an edit to include the non-binning model. Commented Sep 10, 2021 at 16:39
• That's likely to be helpful. There's some structure in the residuals (beyond the obvious lack of fit in the mean and spread). The binning maybe helpful in terms of identifying models. I'd take logs on the y-axis on your first plot to see what's going on. I would also tend to look at a log link rather than the canonical link (although the Gamma's canonical link has a useful interpretation -- 1/time is speed). Commented Sep 11, 2021 at 0:41

library(statmod)