# Understanding a proof

The following is a proof of the existence of M-estimates from the book of Maronna et al. "Robust Statistics". While I understand their strategy, I am having trouble comprehending a step on their proof, namely how taking $\left\| \boldsymbol{\beta} \right\| >b$, implies the inequality $\max_i |y_i -\mathbf{x}_i^{\prime} \boldsymbol{\beta} | \geq b_0$. Could you please help me see how one arrives at this condition? I suspect that I am missing either the geometric picture or some intermediate steps which do not come naturally.

Thank you. This is not a rigorous explanation. I think I understand your confusion. There is very little "regularity" we can assume with arbitrary risk/loss functions. For regular likelihoods, optimization findings are easily available because it's easy to show the score functions are linear in $\beta$. Here, it turns out, existence results are simply a matter of proving that "wilder" and "wilder" choices of $\beta$ will give loss values that spiral into the infinte. Therefore, we can define a ball of $\beta$s for which the risk is continuous and obtain at least one infimum.

$R$ is a function of the residuals. If we define a compact ball for the $\beta$s, the question is: can we produce a similar compact ball for the residuals? If so, we can obtain some optimization results for $R$. Arbitrarily, $2L$ is chosen as a ball for $R$ for which we can indeed construct a ball in $R^p$ for the $\beta$s

It's not written in the proof, but:

$\text{max}_i \left| y_i - x_i \beta \right| \ge ba - \text{max}_i |y_i| \ge b_0$ for all $\| \beta \| > b$

Recall both the $b$ and $a$ were chosen very small in some sense, so that $b_0$ may be the smallest residual. The $b$ is the boundary of a ball in the parameter space, and the $a$ is a minimax like estimator which tells you how "large" a fitted value may be... the choice of $\|\beta\| = 1$ is arbitrary, you see, since any value of $\beta$ in $R^p$ is at least proportional to some $\beta^**b$ where $\|\beta^*\|=1$. Therefore, any choice of value for the norm of $\beta$ (like 3 or 12 instead of 1) would have been proportional to the $a$ we chose. It simply gives you the "biggest" fitted value.

• Okay but I still do not get why the inequalities hold. This was my question by the way, I understand the strategy but not their means. – JohnK Oct 13 '15 at 0:29
• @JohnK I think it's basically the minimax inequality: $\text{min}_\beta \text{max}_i \| y_i - X_i \beta\| \ge \text{max}_i \min_\beta \|y_i - X_i \beta \|$. Then of course, choosing the $a$ and $b$ as given will give you a lower bound for any possible residual in the ball. Does that help? – AdamO Oct 13 '15 at 19:26
• Α little bit. I am sorry but I am utterly confused with their definitions, their indices and their inequalities, it will take some time to grasp the ideas. – JohnK Oct 13 '15 at 19:47
• @JohnK I'm working it out too :) minimax estimation, though, is very interesting stuff. Remember the least squares estimator is just $\hat{\beta} = \min_\beta \| y - X \beta \|$ where the norm is the $\mathcal{L}_2$ norm. The interesting bit about minimax is: betas in a neighborhood of that beta are also pretty good relatively speaking. Because of the convexity of the norm operator, if you replace the $\mathcal{L}_2$ norm with the $\mathcal{L}_\infty$ norm (max), you get a nice wide open ball of betas that map continuously into spaces defined by general loss functions, so optimization holds. – AdamO Oct 13 '15 at 20:01