# estimation using Newton Raphson

How do I find an estimator for gamma using the newton raphson method?

$$\sum_{i=1}^n \biggl(|e_i|-exp(\sum_{j=1}^p \gamma_jX_{ij})\biggl)^2$$

• We would need more context. What is $e_i$, what is $X_{ij}$, what is your objective... Jun 12, 2020 at 10:30
• e is a vector of the residuals and X is the observations matrix Jun 12, 2020 at 10:34
• it's more of a technical question since I haven't learned matrix differentiation Jun 12, 2020 at 10:38
• So the expression above is what you want to minimize right? Your parameter is $\gamma$? Jun 12, 2020 at 11:21
• yes. the parameter is γ Jun 12, 2020 at 12:48

So let's call $$F(\gamma) = \sum_i(\mid e_i\mid-\exp(\sum_j \gamma_j X_{ij}))^2$$ the objective function you want to minimize.
Minimizing $$F$$ is equivalent to solving $$\frac{\partial F}{\partial\gamma}(\gamma) = 0$$
and Newton Raphson methods approximates the solution to this equation with the iterative procedure on $$\gamma$$ defined by the relation:
$$\gamma^{(t + 1)} = \gamma^{(t)} - \left[\frac{\partial^2F}{\partial{\gamma}\partial{\gamma^T}}(\gamma^{(t)})\right]^{-1} \cdot\frac{\partial F}{\partial \gamma}(\gamma^{(t)})$$
So you just need expression for $$\frac{\partial F}{\partial\gamma}(\gamma)$$ and $$\frac{\partial^2 F}{\partial\gamma \partial\gamma^T}(\gamma)$$
Using common derivation rules you get that $$\frac{\partial F}{\partial\gamma}(\gamma) = -\sum_i\left[\mid e_i\mid-\exp(\sum_j \gamma_j X_{ij})\right]\exp(\sum_j \gamma_j X_{ij})X_{i.}$$ where $$X_{i.} = (X_{i1}, X_{i2},...)^T$$. And
$$\frac{\partial F}{\partial\gamma}(\gamma) = -\sum_i\left[\mid e_i\mid- 2 \exp(\sum_j \gamma_j X_{ij})\right]\exp(\sum_j \gamma_j X_{ij})X_{i.}X_{i.}^T$$