4
$\begingroup$

I have the following regression model $$ p_i = x'_i\beta +s \varepsilon_i $$ with sample size $n \approx 150$ and 4 independent variables.

I have reason to believe and $\varepsilon_i$ is distributed iid standard cauchy. So I interpret the above regression as a sort of median regression and solve for $\beta$ and $s$ via maximizing the following log likelihood

$$ \hat\ell(\beta,s\mid x_1,\dotsc,x_n)=- n \log (s \pi) - \sum_{i=1}^n \log \left(1 + \left(\frac{p_i - x'_i\beta}{s}\right)^2\right) $$

The first order conditions are $$ \sum_{i=1}^n \frac{p_i - x'_i\beta}{s^2 + [p_i - x'_i\beta]^2} = 0 $$ $$ \sum_{i=1}^n \frac{s^2}{s^2 + [p_i - x'_i\beta]^2} - \frac{n}{2} = 0 $$

I am fine with numerical optimization (in R using optim or something). But I have a few concerns

  1. I know I am not guaranteed a unique solution in general..at least after seeing MLE of the location parameter in a Cauchy distribution, but how should I go about verifying that my output is actually the global max? I have already run my model in R, and tried random start values which always yeild answers that are identical or very close. I suppose I could also use start values from a preliminary OLS or non-parametric median regression?
  2. Even if I find the global max and the cauchy specification is correct, can I be guaranteed the asymptotic standard errors are good approximations for the finite sample of 150? I get very significant results for this model. I bootstrapped the above and found much larger standard errors which makes everything insignificant :(. But even then my sample size is only 150 and I am bootstrapping Cauchy distributed data, so the bootstrap may not be so reliable either?
  3. I suppose the industry standard alternative is the standard non-parametric median regression with bootstrapped standard errors i.e.

$$ \underset{\beta}{arg min} \left[-\sum_{p_{i}<x'_i\beta}(p_{i}-x'_i\beta)+\sum_{p_{i}\geq x'_i\beta}(p_{i}-x'_i\beta) \right] $$

Would this be a better idea even if I am confident about my Cauchy assumption? Again, I only have 150 obs.

Long story short. I like the Cauchy for theoretical reasons and because it gives me significant coefficients. On the other hand, I am concerned because the non-parametric bootstrap gives higher standard errors implying insignificance. I also don't know how well I can trust the local maxima to be global during numeric optimization. Thank you for all your help.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ I wouldn't use OLS values as starting points; they might work well some of the time but you're risking ending up in a local optimum far from one you'd like to be in. You want a robust regression procedure. If x's may be outlying as well, you'll also want something robust to influential outliers for starting values. $\endgroup$
    – Glen_b
    Commented Apr 15, 2015 at 4:12
  • $\begingroup$ @Glen_b I could use a quantile regression (for the 50th quantile) to estimate the start values of the $\beta$ at least...is that a better idea? I am also considering just going with the quantile regression instead of the Cauchy. However, based on the bootstrapping I have already done, I doubt my estimates will be significant. I suppose it's worth the shot though. I also prefer the Cauchy for theoretical reasons. But if the model is just too fragile, I would be fine switching over to a non-parametric method and accepting insignificant coefficients. $\endgroup$
    – Zachary Blumenfeld
    Commented Apr 15, 2015 at 4:36
  • 1
    $\begingroup$ Certainly $L_1$ regression (i.e. 0.50 quantile) would better than OLS for starting points. However, it isn't robust to influential points (that may not be a concern in your application, I don't know). For the regression itself you might consider some other redescending M-estimator in robust regression in place of likelihood for the Cauchy perhaps, or possibly regression with t-errors (of which Cauchy is a special case) - $t_5$ seems a popular choice in some applications. $\endgroup$
    – Glen_b
    Commented Apr 15, 2015 at 6:07

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.