The model is linear $y_i = a\cdot x_i + b + e_i,~ i = 1,2,\ldots,N $. It is given that the noise is heavy tailed. However the distribution of noise conditional on $x$ is the same for all data points. My question is that how should I model the data generating process? Should I use a Student-t distribution for the noise process? Should I use M estimator in R? Facts: 1. OLS is not to be used. 2. 2. The noise distribution can depend on $x$, but is independent across samples. 3. conditioned on the value of $x$, noise's mean is 0. More clarification: the distribution of noise at $x$, i.e., $N(x)$ is a function of $x$. However for different data samples $i=1,2,\ldots,M$, the distribution of noise samples remains the same.
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ Is this an empirical problem (i.e. fitted model with assumptions being violated) or a theoretical (math) problem? $\endgroup$– JonSep 21, 2016 at 23:44
-
1$\begingroup$ I am given a data set containing $x_i,y_i$ and I have to estimate $a,b$ $\endgroup$– user131929Sep 22, 2016 at 0:50
-
$\begingroup$ See stats.stackexchange.com/questions/66173/… stats.stackexchange.com/questions/154489/… stats.stackexchange.com/questions/259772/… stats.stackexchange.com/questions/26235/… $\endgroup$– kjetil b halvorsen ♦Feb 24, 2017 at 8:20
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
You could use iterative feasible generalized least squares.
Start by setting weights for each datapoint to 1, i.e. no weighting, and use the following algorithm:
- Fit a weighted regression model for each dataset using weights.
- Create a single dataset combining squared residuals/errors, $e_i^2$ and their respective $x$ values.
- Fit $e_i^2 = a\cdot x_i + b$. If the noise is zero mean, $e_i^2$ is equal to the variance of the error at $x_i$.
- Update your weights with the squared errors prediction model
- Go back to 1 until convergence.
-
1$\begingroup$ For the improvement of the community, please describe negative votes. $\endgroup$ Feb 24, 2017 at 17:03