How do I find an estimator for gamma using the newton raphson method?

$$\sum_{i=1}^n \biggl(|e_i|-exp(\sum_{j=1}^p \gamma_jX_{ij})\biggl)^2 $$

  • $\begingroup$ We would need more context. What is $e_i$, what is $X_{ij}$, what is your objective... $\endgroup$ Jun 12, 2020 at 10:30
  • $\begingroup$ e is a vector of the residuals and X is the observations matrix $\endgroup$
    – MC1325
    Jun 12, 2020 at 10:34
  • $\begingroup$ it's more of a technical question since I haven't learned matrix differentiation $\endgroup$
    – MC1325
    Jun 12, 2020 at 10:38
  • $\begingroup$ So the expression above is what you want to minimize right? Your parameter is $\gamma$? $\endgroup$
    – Pohoua
    Jun 12, 2020 at 11:21
  • $\begingroup$ yes. the parameter is γ $\endgroup$
    – MC1325
    Jun 12, 2020 at 12:48

1 Answer 1


So let's call $$F(\gamma) = \sum_i(\mid e_i\mid-\exp(\sum_j \gamma_j X_{ij}))^2$$ the objective function you want to minimize.

Minimizing $F$ is equivalent to solving $$\frac{\partial F}{\partial\gamma}(\gamma) = 0$$

and Newton Raphson methods approximates the solution to this equation with the iterative procedure on $\gamma$ defined by the relation:
$$\gamma^{(t + 1)} = \gamma^{(t)} - \left[\frac{\partial^2F}{\partial{\gamma}\partial{\gamma^T}}(\gamma^{(t)})\right]^{-1} \cdot\frac{\partial F}{\partial \gamma}(\gamma^{(t)})$$

So you just need expression for $\frac{\partial F}{\partial\gamma}(\gamma)$ and $\frac{\partial^2 F}{\partial\gamma \partial\gamma^T}(\gamma)$

Using common derivation rules you get that $$\frac{\partial F}{\partial\gamma}(\gamma) = -\sum_i\left[\mid e_i\mid-\exp(\sum_j \gamma_j X_{ij})\right]\exp(\sum_j \gamma_j X_{ij})X_{i.}$$ where $X_{i.} = (X_{i1}, X_{i2},...)^T$. And

$$\frac{\partial F}{\partial\gamma}(\gamma) = -\sum_i\left[\mid e_i\mid- 2 \exp(\sum_j \gamma_j X_{ij})\right]\exp(\sum_j \gamma_j X_{ij})X_{i.}X_{i.}^T$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.