2
$\begingroup$

I have been trying to estimate parameters of a poisson regression. I am using Newton Raphson method. This method requires that the inverse of Hessian be computed to obtain, updates to beta vector. The text book that I am using defines the Hessian as $$-\sum x'_{i}xe^{x'_i\beta}$$ This doesn't seem to be a $p \times p$ matrix. Here $X$ is $n \times p$ design matrix and $e^{x_i\beta}$ is the mean function and when evaluated it will be a $n \times 1$ vector.

The hessian according to this definition seems to be $n \times 1$ vector. How can we then compute inverse of this vector? What is the correct formula for Hessian?

$\endgroup$
  • 2
    $\begingroup$ You do not have to compute the inverse of the hessian. Instead, you need to solve a system of linear equations in which the hessian appears! $\endgroup$ – Matthew Drury Jan 13 '18 at 18:39
2
$\begingroup$

Based on the notation "$x_i' \beta$" I can infer that $x_i$ a column vector such that $x_i'$ is the $i$th row of the design matrix $X$. Then I suspect the authors meant $$- \sum e^{x_i' \beta} x_i x_i'.$$ Since $x_i x_i'$ is a $p \times p$ matrix, the dimensions make sense.


You can check that this is the same as $$-X' W X = - \begin{bmatrix} | & & | \\ x_1 & \cdots & x_p \\ | & & |\end{bmatrix} \begin{bmatrix} e^{x_1'\beta} \\ & \ddots \\ && e^{x_p'\beta} \end{bmatrix} \begin{bmatrix} - & x_1' & - \\ &\vdots\\ - & x_p' & - \end{bmatrix}$$ from your answer. (Note that you forgot a minus sign.)

$\endgroup$
  • $\begingroup$ I think the negative sign is correct if you are taking the Hessian of the log-likelihood function (which you would maximize). However, if you were instead minimizing the negative log-likelihood (as common in some fields) then the sign would be flipped. $\endgroup$ – Alex Williams Jun 13 '18 at 3:12
1
$\begingroup$

I was able to figure out the correct formulation of Hessian in poisson regression. The hessian is defined as $H=X^TWX$, here $X$ is the design matrix and $W$ is a diagonal matrix where diagonal entries are the $e^{xi\beta}$ Hessian has to be a square matrix as its inverse is used in parameter updates and also used for computing the covariance matrix.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.