2
$\begingroup$

I'll use the data azdrg112 from COUNT package. The los will be the response variable while the remaining three variables gender, type1 and age75 are the explanatory variables. Here is the plot of the response

install.packages('COUNT')
library(COUNT)
data("azdrg112")
barplot(table(azdrg112$los))

enter image description here

The responses seem to be drawn from the Poisson distribution. I modelled it and calculated the absolute mean square error as

pois_model = glm(los ~ ., data = azdrg112, family = poisson)
pois_pred = predict(pois_model, azdrg112, type = 'response')
mean(abs(pois_pred - azdrg112$los)) # 2.330834

I then tried using Linear Regression

gaus_model = glm(los ~ ., data = azdrg112, family = 'gaussian')
gaus_pred = predict(gaus_model, azdrg112, type = 'response')
mean(abs(gaus_pred - azdrg112$los)) # 2.343426

I expected that the Poisson Regression would be a better fit then yield a smaller error but why both models output the same error?

$\endgroup$
4
  • 1
    $\begingroup$ Have you tried this exercise with a dataset where the predictors are not all binary variables? $\endgroup$
    – dipetkov
    Commented Sep 14, 2023 at 7:19
  • $\begingroup$ Yeah, I tried with the data badhealth, also in the same package but it's still the same problem. $\endgroup$
    – Juan
    Commented Sep 14, 2023 at 7:26
  • $\begingroup$ I forgot that the formula los ~ . doesn't add the interactions. Fit the saturated model with los ~ gender * type1 * age75 and you'll get the same predictions with both glm with the poisson family and with lm even though the coefficient estimates are not the same (because their interpretation in the models is different). $\endgroup$
    – dipetkov
    Commented Sep 14, 2023 at 7:38
  • $\begingroup$ I tried but what does it suggest? $\endgroup$
    – Juan
    Commented Sep 14, 2023 at 7:56

1 Answer 1

5
$\begingroup$

Your expectation is wrong. It's all about evaluation metrics and loss functions:

  • Poisson regression minimizes the average unit Poisson deviance.
  • lm() minimizes the mean-squared error (MSE).

Thus: If you evaluate the models with MSE, you can expect lm() to "win". Similarly, if you evaluate with mean unit Poisson deviance, glm() will win.

Now enters Juan the game and chooses mean absolute error as evaluation metric. Neither of the two models tried to minimize this, so either of them might win.

Try: Run a linear median regression via quantreg::rq(...). I'd expect it to do better because it was fitted to minimize mean absolute error.

$\endgroup$
7
  • 1
    $\begingroup$ Thank you for answer, I was researching for a while but I still don't get it. Do you mean the Poisson Regression and the Linear Regression have different loss functions and use different metrics to solve the problem? Also, I tried quantreg::rq(los ~ ., data = azdrg112) which gave me 2.246941 as its MAE, should I consider it as an error reduction or it's just due to chance?. And what do you mean by "enters Joan the game"? $\endgroup$
    – Juan
    Commented Sep 14, 2023 at 8:11
  • $\begingroup$ You used a metric. The two models minimize their respective average loss, which are completely unrelated to your metric. $\endgroup$
    – Michael M
    Commented Sep 14, 2023 at 8:13
  • $\begingroup$ So if my goal is prediction and I use MAE as the metric, I should use a method that has an average loss related to my metric? For example a method whose loss function is defined as sum | predicted values - true values | $\endgroup$
    – Juan
    Commented Sep 14, 2023 at 8:20
  • 1
    $\begingroup$ There is a very slight difference in the MAEs and the predictions are not identical but I'd say very close to each other. The same phenomenon was seen in other datasets that I tried. $\endgroup$
    – Juan
    Commented Sep 14, 2023 at 8:52
  • 1
    $\begingroup$ That seems okay then, thx! $\endgroup$
    – Michael M
    Commented Sep 14, 2023 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.