Timeline for Can the empirical Hessian of an M-estimator be indefinite?
Current License: CC BY-SA 2.5
17 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 28, 2011 at 16:20 | comment | added | whuber♦ | @mpiktas It's a nice exposition; I wish I could vote it up again! One thing, though: when you find a Hessian to be indefinite, it's easy to check whether the solution is on the boundary. If not, don't give up and conclude the "model may be inappropriate"! It's more likely the minimizing algorithm is not a good one for the problem. Furthermore, in your case the model is obviously appropriate: it is identical to the model that produced the data. The difficulty is that (due to the high error variance) it is essentially a one-parameter problem; another parameter is almost unidentifiable. | |
| Feb 25, 2011 at 11:25 | comment | added | mpiktas | @whuber, I rewrote the answer. Also I included the data, so it will be easier to reproduce. I also fixed the bug which was present in the previous version. | |
| Feb 25, 2011 at 11:23 | history | edited | mpiktas | CC BY-SA 2.5 |
rewrote the answer, fixed the bug. I really really hope that this is the last revision.
|
| Feb 24, 2011 at 17:41 | comment | added | whuber♦ | @mpiktas +1 for the idea, even though it seems there are problems with it. I don't follow your first paragraph: it seems to be self-contradictory. Maybe there's a "not" missing somewhere? I am unable to reproduce the results of your example. I do find that in many cases it is difficult to find the global minimum and, occasionally, that failure can lead to a Hessian with a small negative eigenvalue. In thousands of simulations, using a non-derivative based minimizer in Mathematica, I always obtain positive definite Hessians. | |
| Feb 24, 2011 at 15:12 | history | edited | mpiktas | CC BY-SA 2.5 |
added the sanity test
|
| Feb 24, 2011 at 14:57 | comment | added | mpiktas | @Jyotirmoy Bhattacharya, I've updated my answer with more details, hope it explains my intuition which motivated the initial answer. | |
| Feb 24, 2011 at 14:54 | history | edited | mpiktas | CC BY-SA 2.5 |
added note about R
|
| Feb 24, 2011 at 10:07 | comment | added | Jyotirmoy Bhattacharya | @mptikas. Neither Wooldridge nor I are claiming that the Hessian has to be positive definite everywhere. My claim is that for an interior maximum the empirical Hessian has to be positive semidefinite as a necessary condition of a smooth function reaching its maximum. Wooldridge seems to be saying something different. | |
| Feb 18, 2011 at 13:42 | comment | added | mpiktas | @cardinal, I fixed my wording. Now it should be ok. Thanks for pointing out the problem. | |
| Feb 18, 2011 at 13:38 | history | edited | mpiktas | CC BY-SA 2.5 |
fixed according to comments
|
| Feb 18, 2011 at 13:26 | comment | added | mpiktas | @cardinal, yes you are right. Wooldridge uses $w$ for consistency reasons, $y$ and $x$ is reserved for response and predictors throughout the book. In this example $w=(x,y)$. | |
| Feb 18, 2011 at 13:24 | comment | added | cardinal | @mpiktas, I'm not quite sure how to interpret your first sentence due to the wording. I can see two ways, one that I'd call correct and the other I wouldn't. Also, strictly speaking, I don't agree with the second sentence in your first paragraph. As I've shown above, it is possible to be at a local minimum in the interior of the parameter space without the Hessian being positive definite. | |
| Feb 18, 2011 at 13:23 | history | edited | mpiktas | CC BY-SA 2.5 |
fixed an error
|
| Feb 18, 2011 at 13:17 | comment | added | cardinal | @mpiktas, That's some interesting notation there (I know it's not yours). A $w$ on the left-hand side and $y$ and $x$ on the right-hand side. I'm guessing $w = (x,y)$ or something like that. Also, I'm assuming the squaring should be happening to $y - m(x,\theta)$ and not just to $m(x,\theta)$. No? | |
| Feb 18, 2011 at 12:50 | history | edited | mpiktas | CC BY-SA 2.5 |
added 48 characters in body
|
| Feb 18, 2011 at 9:39 | history | edited | mpiktas | CC BY-SA 2.5 |
added 1 characters in body
|
| Feb 18, 2011 at 8:56 | history | answered | mpiktas | CC BY-SA 2.5 |