6
$\begingroup$

I have a model of the form $y_i = a*x_i + b + e_i$. The error terms $e_i$ are independently drawn from a distribution that depends on $x$ as well as on global parameters; however, the noise has conditional mean $0$ given $x$. The goal is to formulate models and derive good estimators for the parameters $a$ and $b$ based on sample $(x; y)$.

Can anyone suggest 1/2 models for this problem?


Edit: Since $E[e|x]=0$ implies $E[e]=0$. So the only two models I can think of are:

  1. $e \sim N(0, x^2 \sigma^2)$, and
  2. $e \sim t(0,x^2,v)$, where in the 2nd model, I am considering non-standardized $t$ distribution with scale parameter $x^2$ and $df=v$.

My question is: Are the 2 models I suggested correct and can there be any more models?

$\endgroup$
  • 1
    $\begingroup$ How is it that the errors are correlated with X & have mean 0 conditional on X? Are you saying the variance depends on X? $\endgroup$ – gung - Reinstate Monica Oct 29 '15 at 22:42
1
$\begingroup$

You could use iterative feasible generalized least squares.

Start by setting weights for each datapoint to 1, i.e. no weighting.

  1. Fit a weighted regression model for each dataset using weights.
  2. Create a single dataset combining residuals/errors and their respective x values.
  3. Fit $e_i^2 = a\cdot x_i + b$. If the noise is zero mean, $e_i^2$ is equal to the variance of the error at $x_i$.
  4. Update your weights with the squared errors prediction model
  5. Go back to 1 until convergence.
$\endgroup$
0
$\begingroup$

If the distribution of your error term can be described by two independent parameters, for example a Gaussian distribution with mean independent of the variance then your problem is familiar. Since $E(error|x)=0$ we can only have the variance/scale of your error depending on $x$. In that case your regression model has heteroskedascity. GLS (Generalized Least Squares) are suitable for this problem as opposed to LS (Least Squares) who assume homoskedascity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.