Questions tagged [least-absolute-deviations]
A regression that minimizes the sum of absolute errors (instead of the sum of squared errors).
20
questions with no upvoted or accepted answers
7
votes
0
answers
247
views
Is there any geometric intuition on least absolute deviation regression?
There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.)
Is there similar geometric intuition for least ...
- 34.9k
4
votes
0
answers
174
views
Most efficient LAD solver
What is the most efficient way to solve linear Least absolute deviation regression problem?
I know it can be solved using linear programming, is there a better/faster method?
Edit: I'm interested ...
- 183
4
votes
0
answers
540
views
Interpretation of the least absolute deviations linear regression coefficient
In linear regression of $y$ onto $x$, one finds a $\beta_0$ and $\beta_1$ minimizing $\sum \|y - (\beta_1 x + \beta_0)\|^2$. One can show that
$$\beta_1 = \rho(x,y) \frac{\sigma(y)}{\sigma(x)},$$
...
- 191
3
votes
0
answers
789
views
Why is there no improvement when training Xgboost with pseudo-Huber loss?
In this StackOverflow post I asked if there was something wrong with my syntax when training an XGboost model (in R) with the native pseudo-Huber loss ...
- 1,021
2
votes
0
answers
53
views
Asymptotic for LAD - Problem 16, p.84 Van der Vaart
From Van der Vaart's Asymptotics Statistics, we have the derivation of the asymptotics for the least square regression (Example 5.27). Now, the problem 16 of the same section regards the asymptotics ...
- 137
2
votes
0
answers
93
views
Efficiency of OLS versus Quantile regression estimator
If I have a linear model
$ y_i = x_i'\boldsymbol\beta + \epsilon_i $
and I assume that OLS estimator of $\boldsymbol\beta$ is unbiased and consistent and Least absolute deviation (LAD) estimator of $...
- 147
2
votes
0
answers
50
views
Comparing the robustness least absolute deviation with OLS
My professor told us that the OLS estimator can be influenced by outliers because
$$\hat{\beta}_{OLS}=\text{argmin}\left\lVert y - X^T \beta\right\rVert_2^2 $$
implies that
the first order ...
- 3,893
1
vote
0
answers
24
views
Is Centering Data Around Their Medians in Least Absolute Deviation Regression Model (No Intercept), a Good Robust Practice For Smaller Data Sets?
Per the regression model:
$\mathbf{y} = f(\mathbf{x},\mathbf{\beta}) + \mathbf{\epsilon}$
Where the $\beta$ estimate of LAD regression is given by:
$ \hat{\beta}_{LAD} = \text{argmin}_{ b} \sum_{i=1}^...
- 1,922
1
vote
0
answers
31
views
Generalisation of Absolute Deviation and Least Squares Deviation: Statistical Use?
This linked answer defines Least Absolute Deviation (LAD) and Least Squares Deviation (LSD), as follows:
$$ LAD = \Sigma^n_{i=1} |y_i-f(x_i)| $$
And:
$$ LSD = \Sigma^n_{i=1} (y_i-f(x_i))^2 $$
...
- 133
1
vote
0
answers
492
views
Consistency of least absolute deviation estimator (LAD) vs OLS
If we know that OLS estimator of beta in linear regression model is unbiased and consistent and we don't have any further assumptions (on errors or anything), will LAD estimator of beta be also ...
- 147
1
vote
0
answers
16
views
Why would the sequence $\hat{x}_{n+1} = \left( A^T E(\hat{x}_n) A\right)^{-1} A^T E(\hat{x}_n)b$ from Least absolut deviation regression converge
Consider the problem
$$
\min_x f(x) = \min_x \|b - Ax\|_{1} = \min_x \sum_i \left| b_i - \sum_j a_{ij}x_{j} \right|
$$
where $x,b$ are vectors and $A$ matrix of some dimension.
Then
$$
\frac{\...
- 123
1
vote
0
answers
33
views
Fisher Information in LAD model for ML estimator
Considering an i.i.d. sample from a linear model $y_i=\alpha x_i+u_i$ (both $y$ and $x$ are centered with respect to their means) errors are homoscedastic and are distributed as:
$$u\sim\frac{1}{\sqrt{...
- 257
1
vote
0
answers
1k
views
Is it possible to force least absolute deviations (LAD) regression to return the 'median' value when infinite solutions are possible?
I have a problem where LAD regressions is not giving me a solution as the R package (L1pack) errors whenever there is an infinite number of possible solutions*. ...
- 4,020
1
vote
0
answers
295
views
Why is Coordinate Descent not used to solve Least Absolute Deviation?
I have recently been looking into why Least Absolute Deviation (LAD) is not used in place of OLS for machine learning, and it appears the primary reason is due to difficulty in computing a solution ...
- 111
0
votes
0
answers
35
views
Recover Primal Linear Programming Solution from Dual with LAD Regression?
This link discusses different ways of writing a classic LAD regression with a linear program. The classic way of writing LAD regression ($y = X \beta + r$) as a linear program is
\begin{equation}
\...
0
votes
0
answers
91
views
Orthogonal regression that minimizes absolute error instead of squared error
Fitting a line through 3D points is usually done by orthogonal least squares (aka "total least squares"), i.e., by minimizing the mean squared orthogonal distance between the points and the ...
- 3,930
0
votes
0
answers
266
views
Regression coefficients while minimizing absolute error
I understand that we like working with square error instead of absolute error because it makes the calculus easy. But I was wondering about the parameters of Linear Regression minimized for absolute ...
0
votes
0
answers
115
views
well maintained lad-regression solution
https://www.real-statistics.com/multiple-regression/lad-regression/
I am trying to understand what is the best package (in R or python) for lad-regression.
But the following one says "but least ...
- 542
0
votes
0
answers
89
views
Least Absolute Values Regression
I have the data:
x <- c(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)
y <- c(2.06, 2.12, 2.32, 2.02, 2.76, 3.04, 2.83, 3.15, 3.36, 3.68, 3.96)
I ...
0
votes
0
answers
101
views
Bayesian least absolute deviations
Ordinary Least Squares have a bayesian equivalent. For instance, fitting the linear model
$$ y \sim a + bx + \epsilon $$
using OLS is equivalent to defining the following Bayesian linear model:
$$ y \...
- 170