Linked Questions

28
votes
3answers
4k views

If linear regression is related to Pearson's correlation, are there any regression techniques related to Kendall's and Spearman's correlations?

Maybe this question is naive, but: If linear regression is closely related to Pearson's correlation coefficient, are there any regression techniques closely related to Kendall's and Spearman's ...
17
votes
2answers
4k views

Why does the L2 norm loss have a unique solution and the L1 norm loss have possibly multiple solutions?

http://www.chioka.in/differences-between-l1-and-l2-as-loss-function-and-regularization/ If you look at the top of this post, the writer mentions that L2 norm has a unique solution and L1 norm has ...
11
votes
3answers
2k views

When would least squares be a bad idea?

If I have a regression model: $$ Y = X\beta + \varepsilon $$ where $\mathbb{V}[\varepsilon] = Id \in \mathcal{R} ^{n \times n}$ and $\mathbb{E}[\varepsilon]=(0, \ldots , 0)$, when would using $\...
11
votes
2answers
18k views

Outlier detection using regression

Can regression be used for out lier detection. I understand that there are ways to improve a regression model by removing the outliers. But the primary aim here is not to fit a regression model but ...
8
votes
1answer
2k views

Choice between different robust regressions in R

I'm writing a program for evaluating real estates and I don't really understand the differences between some robust regression models, that's why I don't know which one to choose. I tried ...
2
votes
2answers
3k views

How to optimize a regression by removing 10% “worst” data points?

I would like to remove 10% of my data points (I consider them as outliers) to maximize the R squared. Is there a way to do so efficiently? I know many people suggest not to remove outliers. But in ...
4
votes
1answer
2k views

Quantile regression vs. Li's regression: which should I use, and when?

Is there a general rule of thumb about when robust regression or quantile regression is preferred in the presence of outliers? For example, I have a dataset where the DV exhibits extreme positive ...
1
vote
3answers
4k views

Determining more than one outlier from a data set

I have a data set of repeated observations and I am trying to determine if any of the observations are outliers. The research I've done has only shown methods that would determine if one value (...
2
votes
2answers
1k views

Theil-Sen estimator assumptions

I found by accident the nonparametric Theil-Sen Estimator as a replacement for standard OLS linear Regression. How well does it perform with autocorrelated data, non-normal residuals and ...
4
votes
1answer
1k views

Other ways to find line of “best” fit

The most common methods I've seen to find a line of best fit are Least Squares regression and median-median. Are there other good ways? Is there a way to minimize the absolute value difference and ...
6
votes
1answer
1k views

What is the maximum likelihood/GLM version of least absolute deviations for robust linear regression?

Robust linear regression from minimising the absolute deviationresults in a regression line of medians conditional on covariates, instead of means using the standard least squares methodology: Is ...
6
votes
1answer
1k views

Elastic net: dealing with wide data with outliers

Recently I was working on a dataset with ~300 observations and 1500 predictors. I used the glmnet package in R to fit an elastic net model, which gave me a cross-...
1
vote
0answers
752 views

Use linear regression to detect outliers and leverage points

I want to use linear regression to pre-process the data (e.g find outliers) so that I can use techniques like ANOVA to analyze the data. The goal is not to fit a regression model. I saw two posts ...
2
votes
1answer
103 views

When to check for outliers

When is a good time for checking the outliers? Do I need to check all variables separately before running the regression or I can bring all variables into the model first and then try to find the ...