# Poisson Regression and Hessian

I have been trying to estimate parameters of a poisson regression. I am using Newton Raphson method. This method requires that the inverse of Hessian be computed to obtain, updates to beta vector. The text book that I am using defines the Hessian as $$-\sum x'_{i}xe^{x'_i\beta}$$ This doesn't seem to be a $p \times p$ matrix. Here $X$ is $n \times p$ design matrix and $e^{x_i\beta}$ is the mean function and when evaluated it will be a $n \times 1$ vector.

The hessian according to this definition seems to be $n \times 1$ vector. How can we then compute inverse of this vector? What is the correct formula for Hessian?

• You do not have to compute the inverse of the hessian. Instead, you need to solve a system of linear equations in which the hessian appears! – Matthew Drury Jan 13 '18 at 18:39

Based on the notation "$x_i' \beta$" I can infer that $x_i$ a column vector such that $x_i'$ is the $i$th row of the design matrix $X$. Then I suspect the authors meant $$- \sum e^{x_i' \beta} x_i x_i'.$$ Since $x_i x_i'$ is a $p \times p$ matrix, the dimensions make sense.
You can check that this is the same as $$-X' W X = - \begin{bmatrix} | & & | \\ x_1 & \cdots & x_p \\ | & & |\end{bmatrix} \begin{bmatrix} e^{x_1'\beta} \\ & \ddots \\ && e^{x_p'\beta} \end{bmatrix} \begin{bmatrix} - & x_1' & - \\ &\vdots\\ - & x_p' & - \end{bmatrix}$$ from your answer. (Note that you forgot a minus sign.)
I was able to figure out the correct formulation of Hessian in poisson regression. The hessian is defined as $H=X^TWX$, here $X$ is the design matrix and $W$ is a diagonal matrix where diagonal entries are the $e^{xi\beta}$ Hessian has to be a square matrix as its inverse is used in parameter updates and also used for computing the covariance matrix.