Goodness of fit and which model to choose linear regression or Poisson

I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable.

My questions are

• What are the most important criteria for a good model? What is more/less important? Is it that most or all of the variables will be significant? Is it the prob of "F STATISTIC"? Is it the value of "Adjusted R squared"?

• Second, how can I decide the most appropriate model for the research? Besides patents which are a count variable (so maybe a Poisson count) I have explanatory variables such as return on assets, research and development budget, repeated partner (% not a binary variable), company size (employees) and a couple more. Should I do a linear regression or Poisson?

• Kjetil has given a good detailed answer. A quicker and briefer opinion consistent with his argument is that what you label "second" is the really important question. What you mention first is incidental. – Nick Cox Mar 18 '15 at 19:30

Most important is the logic behind the model. Your variable "number of patents per year" is a count variable, so Poisson regression is indicated. That is a GLM (generalized linear model) with (usually) log link function, while the usual linear regression is a Gaussian GLM with identity link. Here, it is truly the log link function which is most important, more important than the error distribution (Poisson or Gaussian).

The variable "Patents" is an extensive variable: see intensive and extensive properties. For intensive variables, such as temperature, linear models (with identity link) are often appropriate. But with an extensive variable it is different. Think that one of your pharmaceutical companies split into two different companies. Then the patents had to be split among the two new companies. What happens with the covariables, the $x$'s in your regression? Variables like number of employees and RD budget would have to be split too.

Broadly, in this context, an intensive variable is a variable which is independent of company size, while an extensive variable depends (typically, linearly) on company size. So, in a sense, if we have many different extensive variables in the regression equation, we are measuring size effects repeatedly. That seems redundant, so we should try to, when possible, express variables in intensive form, like RD budget per employee (or as percent of total budget), likewise revenue, etc. A variable like number of employees will have to be left as extensive. See @onestop's answer to Dealing with correlated regressors for another discussion of this extensive/intensive variable issue.

Let's look at this algebraically: $P, B, E$ are Patents, Budget (per employee), Employees in the original company, while $P_1, B_1, E_1$ and $P_2, B_2, E_2$ are the corresponding variables after a split. Assume, as above, that $E$ is the only extensive covariable (with $P$, of course, also extensive).

Then, before the split, we have the model, identity link, with random part left out: $$P= \mu+\beta_1 E + \beta_2 B$$ Let the split fractions be $\alpha, 1-\alpha$ so for company 1 after the split we get \begin{align} \alpha P &= \alpha \mu +\alpha\beta_1 E +\alpha\beta_2 B \\[5pt] P_1 &= \alpha\mu + \beta_1 E_1 + \alpha\beta_2 B_1 \end{align} since $P_1=\alpha P, E_1=\alpha E$ but $B_1=B$. Likewise for company two. So the model depends in a quite complicated way on company size, only the regression coefficient on $E$ being independent of company size, size influencing all other parameters. That makes interpretation of results difficult, especially so, if in your data you have companies of varying size, then how are you going to interpret those coefficients? Comparison with other studies based on other data, etc., gets wildly complicated.

Now, let us see if using a log link function can help. Again, we write idealized models without disturbance terms. The variables are as above.

First, the model before the split: $$P = \exp\left(\mu+\beta_1 E + \beta_2 B\right)$$ After the split, for company one, we get: \begin{align} P_1 &= \exp(\log\alpha) \exp\left(\mu+\beta_1 E + \beta_2 B\right) \\[5pt] P_1 &= \exp\left(\log\alpha+\mu+\beta_1 E +\beta_2 B_1 \right) \end{align} This looks almost right, except for one problem, the part of dependency on $E$ doesn't quite work out. So we see that number of employees, the one covariable in extensive form, must be used on a log scale. Then, trying again, we get:

Model before the split: $$P = \exp\left(\mu+\beta_1 \log E + \beta_2 B\right)$$ After the split: \begin{align} P_1 &= \exp(\log\alpha) \exp\left(\hspace{9.5mm}\mu+\beta_1 \log E + \beta_2 B\right) \\[5pt] P_1 &= \exp\left(\log\alpha+\hspace{6mm}\hspace{9.5mm}\mu+\beta_1 \log E +\beta_2 B_1 \right) \\[5pt] P_1 &= \exp\left((1-\beta)\log\alpha+\mu+\beta_1 \log E_1 +\beta_2 B_1\right) \\[5pt] P_1 &= \exp\left(\hspace{31mm}\mu'+\beta_1 \log E_1 +\beta_2 B_1\right) \end{align} where $\mu'$ is a new intercept. Now, we have put the model in a form where all parameters (except the intercept) have an interpretation independent of company size.

That makes interpretations of results much easier, and also comparisons with studies using other data, trends with time, and so on. You cannot achieve this form with parameters with size-independent interpretations with an identity link.

Conclusion: Use a GLM with log link function, maybe a Poisson regression, or negative-binomial, or ... The link function is orders of magnitude more important!

To sum up, when constructing a regression model for a response variable which is extensive, like a count variable.

1. Try to express covariables in intensive form.

2. Covariables which must be left as extensive: log them (the algebra above depends on there being at most one extensive covariable).

3. Use a log link function.

Then, other criteria, such as those based on fit, can be used for secondary decisions, such as the distribution of the disturbance term.

• I thought a Poisson regression was a GLS regression with log link function? – Sideshow Bob Dec 18 '15 at 10:07
• Usually, yes, but you could also consider a Poisson regression with identity (or other, such as square root) link. But my argument here shows that usually, you want the log link. – kjetil b halvorsen Dec 18 '15 at 10:28
• @ kjetil b halvorsen, what is $(1-\beta)$ in correction of $\mu$ term? – garej May 10 at 6:15