3
$\begingroup$

Suppose $y=ax+z$ where $x, y, z$ are random variables with range in $\mathbf R$, $\mathbf E[x]=0$, the probability distribution $p(z|x)$ is

1) normal distribution $N(0,\sigma(x)^2)$ with mean $0$ and standard deviation $\sigma(x)$ as an unknown function of $x$;

2) student t-distribution $t_{\nu(x)}$ with degrees of freedom $\nu(x)$ an unknown function of $x$,

and $a$ is an unknown constant. Suppose $(x_i,y_i)_{i=1}^n$ is a set of tuples of sample observation of $(x,y)$. How do we estimate the following functions?

1) $(a,\sigma(x))$;

2) $(a,\nu(x))$.


Note: This is not the heteroscedasticity problem in the conventional sense where the dispersion parameter depends on the index $i$. The dispersion parameter now depends on the independent variable $x$.

$\endgroup$
  • 1
    $\begingroup$ a little more context would be nice. It would be fairly straightforward to write down the maximum likelihood equations for this, and maybe to solve for the MLE (I haven't tried). Computationally, you could consider this (at least the first, and maybe the second) a generalized least squares problem, and fit it (e.g. with gls() from the nlme package in R. $\endgroup$ – Ben Bolker Feb 22 at 3:46
  • $\begingroup$ @BenBolker: Thank you. I agree that MLE is a way to go. $\endgroup$ – Hans Feb 22 at 8:53
3
$\begingroup$

My guess is you can reasonably estimate $a$ with OLS in (1) and a maybe a more robust estimation in (2) like like IRLS (though maybe OLS might still be ok).

Estimating $\sigma(x)$ and $\nu(x)$ is harder. I think you need to decide how to parameterize or quantize wrt $x$. For instance, if you choose some function with parameters $\theta$, and define your estimate to be $\hat{\sigma}(x)=f(x;\theta)$, then you can numerically fit $\theta$ by maximizing the log-likelihood over the dataset $(x_i, z_i)=(x_i,y_i-ax_i)$. The choice of $f$ represents some level of "prior" over $\sigma(x)$ or $\nu(x)$.

I guess you can also jointly estimate $a$ and $\sigma$ or $\nu$ together by combining the two optimization approaches above in an alternating manner.

Related Links

$\endgroup$
  • $\begingroup$ +1 and thank you for the prompt and informative answer. Let me read through the links. $\endgroup$ – Hans Feb 22 at 8:51
  • $\begingroup$ @Hans hopefully it helps. PS I think you can get/estimate $\sigma$ from robust standard errors in (1). The same might also work for (2). Perhaps see stats.stackexchange.com/questions/275925/… or maybe stats.stackexchange.com/questions/258485/… $\endgroup$ – user3658307 Feb 22 at 13:10
  • $\begingroup$ Sorry for the late acceptance of your answer. Thank you again for your richly informative answer. $\endgroup$ – Hans Apr 14 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.