Linear regression with error dispersion dependent on the independent variable

Question

Suppose $y=ax+z$ where $x, y, z$ are random variables with range in $\mathbf R$, $\mathbf E[x]=0$, the probability distribution $p(z|x)$ is

1) normal distribution $N(0,\sigma(x)^2)$ with mean $0$ and standard deviation $\sigma(x)$ as an unknown function of $x$;

2) student t-distribution $t_{\nu(x)}$ with degrees of freedom $\nu(x)$ an unknown function of $x$,

and $a$ is an unknown constant. Suppose $(x_i,y_i)_{i=1}^n$ is a set of tuples of sample observation of $(x,y)$. How do we estimate the following functions?

1) $(a,\sigma(x))$;

2) $(a,\nu(x))$.

---

Note: This is not the heteroscedasticity problem in the conventional sense where the dispersion parameter depends on the index $i$. The dispersion parameter now depends on the independent variable $x$.

Answer 1

My guess is you can reasonably estimate $a$ with OLS in (1) and a maybe a more robust estimation in (2) like like IRLS (though maybe OLS might still be ok).

Estimating $\sigma(x)$ and $\nu(x)$ is harder. I think you need to decide how to parameterize or quantize wrt $x$. For instance, if you choose some function with parameters $\theta$, and define your estimate to be $\hat{\sigma}(x)=f(x;\theta)$, then you can numerically fit $\theta$ by maximizing the log-likelihood over the dataset $(x_i, z_i)=(x_i,y_i-ax_i)$. The choice of $f$ represents some level of "prior" over $\sigma(x)$ or $\nu(x)$.

I guess you can also jointly estimate $a$ and $\sigma$ or $\nu$ together by combining the two optimization approaches above in an alternating manner.

Related Links

• MLE for $t$-distributed errors

• Estimating parameters of $t$ distributions

• Non-normal OLS residuals (also this one)