1
$\begingroup$

I have a decent understand of OLS regression. Now, what if my observation isn't normally distributed anymore, how can I estimate my parameter of the regression model?

I was trying to estimate the parameter by minimize the negative log-likelihood. I have $$ l(\theta,\gamma) = n\log(\gamma) - \sum \log(\gamma^2 + (x-\theta)^2 ) $$ where $\gamma$ and $\theta$ is scale and location parameter of cauchy distribution. Then, let $ \frac{dl}{d\theta} = 0 , \frac{dl}{d\gamma} = 0 $, I got $$ \sum\frac{x-\theta}{\gamma^2 + (x - \theta)^2} =0 $$ and $$ \sum\frac{\gamma^2}{\gamma^2 + (x - \theta)^2} = n/2 $$

How do you go from $\theta$ and $\gamma$ to $\beta$? Am I on the right track at all?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ You have the location one correct, but the scale one looks (very) slightly wrong (the numerator in the sum is not quite right). See some related questions here and here which discuss the location parameter for a unit scale and may help focus your question $\endgroup$
    – Glen_b
    Commented Oct 10, 2016 at 10:15
  • $\begingroup$ @Glen_b, thanks for the link!. Since I have two variables to solve, I assume Newton-Raphson Method wouldn't work here. $\endgroup$
    – Phume
    Commented Oct 10, 2016 at 17:21
  • 2
    $\begingroup$ Newton Raphson works with multiple variables, but derivatives are vector-valued and second derivatives (and their inverses) become matrices. However if you start with good estimates you should be able to alternate updating your estimate of one and then the other though it may be slower. You can get a good (highly efficient) estimate of $\gamma$ (for $n>2$) by using the approach in Kravchuk&Pollett (there's a working version here people.smp.uq.edu.au/PhilipPollett/papers/ScaleRankCauchy.pdf), but see the section in wikipedia on estimation of parameters ... ctd $\endgroup$
    – Glen_b
    Commented Oct 10, 2016 at 19:02
  • 1
    $\begingroup$ ctd... here; in particular the likelihood when estimating the location can sometimes have multiple local modes and MLE may not be efficient at small sample sizes. $\endgroup$
    – Glen_b
    Commented Oct 10, 2016 at 19:02
  • 2
    $\begingroup$ Possible duplicate of can you give a simple intuitive explanation of IRLS method to find the MLE of a GLM? $\endgroup$
    – kjetil b halvorsen ♦
    Commented Sep 8, 2017 at 15:02

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.