5
$\begingroup$

I do not understand the role of weights in "weighted Poisson regression". What exactly is being weighted? Is it the contribution of the observation to the log-likelihood of the model, or something else?

In the following two popular threads,

Where does the offset go in Poisson/negative binomial regression?

When to use an offset in a Poisson regression?

commentators establish the equivalence between Poisson regression with explicit offset $t_i$ (for exposure time, for example) in equation:

$$\ln(\lambda_i) = \beta_0 + \beta_1 x_{i1} + \dots + \beta_1 x_{iN} + \ln(t_i)$$

and weighted Poisson regression with weights $t_i$ (at least in R):

$$\ln\!\bigg(\frac{\lambda_i}{t_i}\bigg) = \beta_0 + \beta_1 x_{i1} + \dots + \beta_1 x_{iN}$$

By equivalent, one of the threads demonstrates with an example that the estimated coefficients are the same.

However, I don't understand what the weighting in the second regression means? What are the objective functions being optimised in both cases? In the first one is it the normal Poisson log-likelihood: $-\lambda + k \ln(\lambda) - \ln(k!)$?

$\endgroup$
  • $\begingroup$ Do you understand statistically what a weighted linear regression is? Same thing. $\endgroup$ – Hong Ooi Feb 26 '17 at 10:34
  • 1
    $\begingroup$ nope, not precisely. I have seen found this link: stats.stackexchange.com/questions/91386/… which agrees with my understanding of weighted least squares, at least how to implement the objective function. However, the implementation does not actually throw any light on why it works. To be precise, why does penalising the log-likelihood or squared errors by w actually correspond to a model where variances in the errors are not assumed to be consta nt? $\endgroup$ – Alex Feb 26 '17 at 11:07
  • 1
    $\begingroup$ Possible duplicate of When to use an offset in a Poisson regression? $\endgroup$ – kjetil b halvorsen Sep 17 '17 at 13:03
  • 1
    $\begingroup$ Hi @kjetilbhalvorsen. The proposed dupe question does not resolve my questions in the last paragraph. I would be keen to hear why you think it is better/more relevant than the answer by cokes, which explains very explicitly what the reweighting process is doing. $\endgroup$ – Alex Sep 18 '17 at 1:09
  • 1
    $\begingroup$ @Alex: I retracted my close vote. $\endgroup$ – kjetil b halvorsen Sep 18 '17 at 16:59
9
$\begingroup$

This also confused me. I thought, "what is the point of explicitly including an offset instead of just pretending that the response divided by the offset / exposure is the $y$ value?".

You actually get two different loss functions if you do so.

The correct way (use an exposure/offset $s_i$)

Model $\log \lambda_i = \log s_i + \theta^T x$ so that $\lambda_i = s_i e^{\theta^Tx}$. This makes complete sense: the exposure $s_i$ just multiplies the $\hat{\lambda_i}=e^{\hat{\theta}^Tx}$ in a Poisson regression model without different exposures.

We model the random variable $Y$, a response to $x_i$, with a Poisson distribution with parameter $\lambda_i$.

Then the likelihood for $N$ data points is:

$$\prod_{i=1}^N \dfrac{(s_ie^{\theta^Tx})^{y_i}}{y_i!}e^{-s_i e^{\theta^Tx}}$$

The log likelihood $\ell$, keeping only terms that depend on $\theta$ since others will drop out after differentiation:

$$\ell = \displaystyle \sum_{i=1}^N\big[ y_i\theta_Tx_i -s_i e^{\theta^Tx_i}\big]$$

The incorrect way (using $y_i/s_i$ as the y-values)

Now we still model:

$$\log \lambda_i = \log s_i + \theta^T x$$

The difference is that now we assume $y_i/s_i$ has a Poisson distribution. This is essentially what makes the model incorrect. It violates the assumption that $y_i$ has a Poisson distribution. Now you are modeling the rate as having a Poisson distribution. So the likelihood is now:

$$\prod_{i=1}^N \dfrac{(e^{\theta^Tx})^{y_i/s_i}}{(y_i/s_i)!}e^{- e^{\theta^Tx}}$$

[Awkward to have $y_i/s_i$ in the factorial term but it drops out anyway after differentiation of the log likelihood so let's carry on.]

The log likelihood $\hat{\ell}$, keeping only terms that depend on $\theta$ since others will drop out after differentiation:

$$\hat{\ell} = \displaystyle \sum_{i=1}^N\bigg[ \frac{y_i}{s_i}\theta_Tx_i - e^{\theta^Tx_i}\bigg]$$

Conclusion

$\ell$ and $\hat{\ell}$ look strikingly similar, and you might think they are the same, but they are not (you can't just divide by $s_i$ because it is different for every term!)

However, if we consider a weighted Poisson regression when we model $y_i/s_i$ as distributed Poissonian (is that a word?), each data point in the log likelihood gets a weight of $s_i$, then:

$$\hat{\ell}_{{\rm weighted}}=\displaystyle \sum_{i=1}^N s_i[ \frac{y_i}{s_i}\theta_Tx_i - e^{\theta^Tx_i}]$$

is equivalent to $\ell$!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.