2
$\begingroup$

The model is linear $y_i = a\cdot x_i + b + e_i,~ i = 1,2,\ldots,N $. It is given that the noise is heavy tailed. However the distribution of noise conditional on $x$ is the same for all data points. My question is that how should I model the data generating process? Should I use a Student-t distribution for the noise process? Should I use M estimator in R? Facts: 1. OLS is not to be used. 2. 2. The noise distribution can depend on $x$, but is independent across samples. 3. conditioned on the value of $x$, noise's mean is 0. More clarification: the distribution of noise at $x$, i.e., $N(x)$ is a function of $x$. However for different data samples $i=1,2,\ldots,M$, the distribution of noise samples remains the same.

$\endgroup$
-1
$\begingroup$

Fat tailed $e$ does not violate Gauss--Markov. Without more context on your problem or assuming Gauss--Markov doesn't hold, good old OLS is the best linear unbiased estimator.

Are there any other aspects to the problem that you didn't mention?

$\endgroup$
  • $\begingroup$ Following are the problem details: 1. Gaussian linear models are not to be considered because noise can be heavytailed. 2. The noise distribution can depend on $x$, but is independent across samples. 3. conditioned on the value of $x$, noise's mean is $0$. $\endgroup$ – user131929 Sep 22 '16 at 4:32
  • 1
    $\begingroup$ dmsul -- If the distribution is sufficiently far from normal, all linear estimators may be bad. Beating a collection of other bad estimators is a pretty dubious crown. $\endgroup$ – Glen_b Sep 22 '16 at 4:41
  • 1
    $\begingroup$ @user131929 1. Is this for a class? 2. You should include relevant information in the question rather in comments where people may miss it. 3 Your point 2 above contradicts the third sentence of your question. Please clarify the question to remove any ambiguity. 4 Heavy tails in the error distribution can (fairly dramatically) increase the problem with unusual points in x-space. What is known about the arrangement of x's? 5 There are various things one might try to do well at with a problem like this -- what are you trying to achieve, exactly? $\endgroup$ – Glen_b Sep 22 '16 at 4:47
  • $\begingroup$ I have included the comments in the question itself. Regarding your comment, "Your point 2 above contradicts the third sentence of your question." I can't see any contradiction. Point 2 simply implies that the distribution of noise at $x$, i.e., $N(x)$ is a function of $x$. However for different data samples $i=1,2,\ldots,M$, the distribution of noise samples remains the same. $\endgroup$ – user131929 Sep 23 '16 at 6:58
-1
$\begingroup$

You could use iterative feasible generalized least squares.

Start by setting weights for each datapoint to 1, i.e. no weighting, and use the following algorithm:

  1. Fit a weighted regression model for each dataset using weights.
  2. Create a single dataset combining squared residuals/errors, $e_i^2$ and their respective $x$ values.
  3. Fit $e_i^2 = a\cdot x_i + b$. If the noise is zero mean, $e_i^2$ is equal to the variance of the error at $x_i$.
  4. Update your weights with the squared errors prediction model
  5. Go back to 1 until convergence.
$\endgroup$
  • 1
    $\begingroup$ For the improvement of the community, please describe negative votes. $\endgroup$ – rafaelvalle Feb 24 '17 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.