Questions tagged [least-absolute-deviations]
A regression that minimizes the sum of absolute errors (instead of the sum of squared errors).
26
questions
1
vote
1answer
58 views
Does this exist: Recursive Least Absolute Deviation-Regression?
For least squares regression there exists the Recursive Least Squares algorithm which allows to find the least squares solution online.
Does something similar exist for Least Absolute Deviation ...
21
votes
7answers
2k views
Why is using squared error the standard when absolute error is more relevant to most problems? [duplicate]
I recognize that parts of this topic have been discussed on this forum. Some examples:
Is minimizing squared error equivalent to minimizing absolute error? Why squared error is more popular than the ...
1
vote
0answers
17 views
Generalisation of Absolute Deviation and Least Squares Deviation: Statistical Use?
This linked answer defines Least Absolute Deviation (LAD) and Least Squares Deviation (LSD), as follows:
$$ LAD = \Sigma^n_{i=1} |y_i-f(x_i)| $$
And:
$$ LSD = \Sigma^n_{i=1} (y_i-f(x_i))^2 $$
...
1
vote
1answer
41 views
Reference for doing linear regression with mean absolute deviation?
I am looking for a resource that goes over how to derive the coefficients for a linear regression model while minimizing the mean absolute deviation. I am hoping for both a mathematical and ...
2
votes
0answers
33 views
Efficiency of OLS versus Quantile regression estimator
If I have a linear model
$ y_i = x_i'\boldsymbol\beta + \epsilon_i $
and I assume that OLS estimator of $\boldsymbol\beta$ is unbiased and consistent and Least absolute deviation (LAD) estimator of $...
2
votes
0answers
173 views
Consistency of least absolute deviation estimator (LAD) vs OLS
If we know that OLS estimator of beta in linear regression model is unbiased and consistent and we don't have any further assumptions (on errors or anything), will LAD estimator of beta be also ...
2
votes
0answers
26 views
Comparing the robustness least absolute deviation with OLS
My professor told us that the OLS estimator can be influenced by outliers because
$$\hat{\beta}_{OLS}=\text{argmin}\left\lVert y - X^T \beta\right\rVert_2^2 $$
implies that
the first order ...
1
vote
1answer
51 views
Convergence of weighted regression solution to the solution of L1 regression
I have read in this paper that the weighted regression solution converges to the solution of L1 regression,
for weights, $$W_i=1/|y_i-\hat y_i|$$
I worked this out but unfortunately, I lost.
Could ...
1
vote
0answers
31 views
least absolute deviation version of deming [closed]
Based on what I know, deming minimizes the sum of square of perpendicular distance to the regression line. Is there a package in R that can run regression that minimize the sum of absolute value of ...
1
vote
0answers
16 views
Why would the sequence $\hat{x}_{n+1} = \left( A^T E(\hat{x}_n) A\right)^{-1} A^T E(\hat{x}_n)b$ from Least absolut deviation regression converge
Consider the problem
$$
\min_x f(x) = \min_x \|b - Ax\|_{1} = \min_x \sum_i \left| b_i - \sum_j a_{ij}x_{j} \right|
$$
where $x,b$ are vectors and $A$ matrix of some dimension.
Then
$$
\frac{\...
1
vote
4answers
700 views
How regression trees split, when all the Features and target have only continuous values
Can anyone please explain how splitting is performed in regression trees when we only have continuous features. I have referred to different papers, but all I could find is formulas or theorems.
Can ...
4
votes
0answers
74 views
Most efficient LAD solver
What is the most efficient way to solve linear Least absolute deviation regression problem?
I know it can be solved using linear programming, is there a better/faster method?
Edit: I'm interested ...
2
votes
1answer
1k views
Is there any library for least absolute deviation (LAD) regression with regularization terms?
We know that LASSO and ridge and ElasticNet all apply regularization terms on the coefficients of least squares regression. However, I have not yet found any R / python libraries that compute ...
6
votes
0answers
199 views
Is there any geometric intuition on least absolute deviation regression?
There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.)
Is there similar geometric intuition for least ...
3
votes
0answers
327 views
Interpretation of the least absolute deviations linear regression coefficient
In linear regression of $y$ onto $x$, one finds a $\beta_0$ and $\beta_1$ minimizing $\sum \|y - (\beta_1 x + \beta_0)\|^2$. One can show that
$$\beta_1 = \rho(x,y) \frac{\sigma(y)}{\sigma(x)},$$
...
1
vote
1answer
1k views
Can I use gradient descent for Least Absolute Deviation Regression?
I've read in some posts that the absolute function is not differentiable. However, it is not differentiable only at 0. Can I not check the value at each training point to calculate the derivative. ...
3
votes
2answers
1k views
Model that optimizes mean absolute error always gives same prediction
My gradient boosting regression model (GBM) is trained to minimize mean absolute error (MAE) but gives the same prediction for every record on my highly skewed dataset. I believe there is a quick fix ...
1
vote
0answers
29 views
Fisher Information in LAD model for ML estimator
Considering an i.i.d. sample from a linear model $y_i=\alpha x_i+u_i$ (both $y$ and $x$ are centered with respect to their means) errors are homoscedastic and are distributed as:
$$u\sim\frac{1}{\sqrt{...
1
vote
1answer
343 views
For syntax of least absolute deviation from package 'L1pack' [closed]
I am trying to use the least absolute deviation regression from my dataset which has one column of dependent variable and multiple columns of independent variables. I tried using the following syntax ...
1
vote
0answers
998 views
Is it possible to force least absolute deviations (LAD) regression to return the 'median' value when infinite solutions are possible?
I have a problem where LAD regressions is not giving me a solution as the R package (L1pack) errors whenever there is an infinite number of possible solutions*. ...
6
votes
1answer
978 views
What is the maximum likelihood/GLM version of least absolute deviations for robust linear regression?
Robust linear regression from minimising the absolute deviationresults in a regression line of medians conditional on covariates, instead of means using the standard least squares methodology: Is ...
12
votes
2answers
4k views
How to solve least absolute deviation by simplex method?
Here is the least absolute deviation problem under concerned: $ \underset{\textbf{w}}{\arg\min} L(w)=\sum_{i=1}^{n}|y_{i}-\textbf{w}^T\textbf{x}|$. I know it can be rearranged as LP problem in ...
1
vote
0answers
196 views
Why is Coordinate Descent not used to solve Least Absolute Deviation?
I have recently been looking into why Least Absolute Deviation (LAD) is not used in place of OLS for machine learning, and it appears the primary reason is due to difficulty in computing a solution ...
3
votes
1answer
874 views
$R^2$ for least absolute deviation regression
I know that $R^2$ is for the least square regression. Is there an analogous measure of fit to $R^2$ in LAD (Least Absolute Deviations) regression?
Here I am concerned with the "fitting quality".
1
vote
0answers
78 views
What is the standard measure of fit quality for least absolute deviation regression (the analog of $R^2$) [duplicate]
When one runs an OLS regression, one often prefers to look at $R^2$ rather than the mean squared error, though both measure goodness of fit. $R^2$ is dimensionless and has some advantageous properties:...
8
votes
1answer
263 views
When does Least Square Regression (LSQ) line equal to Least Absolute Deviation (LAD) line?
I have the following question at hand.
Suppose $(x_1,y_1),(x_2,y_2),\cdots,(x_{10},y_{10})$ represents a set of bi-variate observations on $(X,Y)$ such that $x_2=x_3=\cdots =x_{10}\ne x_1.$ Under ...
