How do I find an estimator for gamma using the newton raphson method?
$$\sum_{i=1}^n \biggl(|e_i|-exp(\sum_{j=1}^p \gamma_jX_{ij})\biggl)^2 $$
5
-
$\begingroup$ We would need more context. What is $e_i$, what is $X_{ij}$, what is your objective... $\endgroup$– Álvaro Méndez CivietaJun 12, 2020 at 10:30
-
$\begingroup$ e is a vector of the residuals and X is the observations matrix $\endgroup$– MC1325Jun 12, 2020 at 10:34
-
$\begingroup$ it's more of a technical question since I haven't learned matrix differentiation $\endgroup$– MC1325Jun 12, 2020 at 10:38
-
$\begingroup$ So the expression above is what you want to minimize right? Your parameter is $\gamma$? $\endgroup$– PohouaJun 12, 2020 at 11:21
-
$\begingroup$ yes. the parameter is γ $\endgroup$– MC1325Jun 12, 2020 at 12:48
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
So let's call $$F(\gamma) = \sum_i(\mid e_i\mid-\exp(\sum_j \gamma_j X_{ij}))^2$$ the objective function you want to minimize.
Minimizing $F$ is equivalent to solving $$\frac{\partial F}{\partial\gamma}(\gamma) = 0$$
and Newton Raphson methods approximates the solution to this equation with the iterative procedure on $\gamma$ defined by the relation:
$$\gamma^{(t + 1)} = \gamma^{(t)} - \left[\frac{\partial^2F}{\partial{\gamma}\partial{\gamma^T}}(\gamma^{(t)})\right]^{-1} \cdot\frac{\partial F}{\partial \gamma}(\gamma^{(t)})$$
So you just need expression for $\frac{\partial F}{\partial\gamma}(\gamma)$ and $\frac{\partial^2 F}{\partial\gamma \partial\gamma^T}(\gamma)$
Using common derivation rules you get that $$\frac{\partial F}{\partial\gamma}(\gamma) = -\sum_i\left[\mid e_i\mid-\exp(\sum_j \gamma_j X_{ij})\right]\exp(\sum_j \gamma_j X_{ij})X_{i.}$$ where $X_{i.} = (X_{i1}, X_{i2},...)^T$. And
$$\frac{\partial F}{\partial\gamma}(\gamma) = -\sum_i\left[\mid e_i\mid- 2 \exp(\sum_j \gamma_j X_{ij})\right]\exp(\sum_j \gamma_j X_{ij})X_{i.}X_{i.}^T$$
