1
$\begingroup$

I got some data samples (a,b) and I am trying to calculate correlation between a and b. As I am new in this type of analysis, I have calculated Rsquared with linear regression method and got 0.5 as the result. However, in my graph I could see one clear outlier and if I exclude it manually, Rsquared increases from 0.5 to 0.748 which is significant increase.

In order not to do it manually, is there some method that somebody would recommend, that does robust fit and identifies outlier and excludes them from final R square calculation?

Thanks!

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
0
$\begingroup$

Yes, to keep it simple, you can request that the standardized residuals be determined during the regression run. Any standardized residual less than -1.96 or greater than 1.96 will be an outlier. The question you are asking addresses the topic of Regression Diagnostics, which focuses on different types of residuals, and goodness-of-fit (e.g. $R^2$) of a regression model.

Residuals are basically observations that are "overly influential" on the regression model.

One of my favorite types of residuals are called leverage residuals, which are observations whose $x/y$ values "pull too much" on the fitted regression line. This can occur with e.g. a cluster of observations that are far removed from the rest of the bulk of the data. Cook's distance is another type of residual, as well as deletion residuals, studentized residuals, DFFITS, and DFBETAS. But for starters, you should focus on standardized using the criterion of $\pm 2$.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks! Ill look at it now. Does Regression diagnostics and standardized residuals have anything to do with RANSAC robust regression? I have found out that possibly I can try to use RANSAC but due to my low understanding of the matter I am not sure how to apply RANSAC on my original dataset as all the scripts I found are related to the training sets. $\endgroup$
    – sergio
    Commented Apr 12, 2021 at 4:18
  • $\begingroup$ Don't know about RANSAC, but robust methods usually focus on methods to reduce inflated standard errors, s.e., of regression coefficients. Statistical significance for each $j$th regression coefficient is equal to a t-statistic, where $t=\beta_j / s.e.(\beta_j)$ and robust methods usually try to reduce $s.e.(\beta_j)$. Thus, not related to outliers. $\endgroup$
    – user318288
    Commented Apr 12, 2021 at 4:23
  • $\begingroup$ Thanks! I will take a look now if I can apply it on my data. I have would out from here [medium.com/@angel.manzur/got-outliers-ransac-them-f12b6b5f606e] that I could possibly also try RANSAC but need to find a away how to apply it on my data :) $\endgroup$
    – sergio
    Commented Apr 12, 2021 at 4:50
  • 1
    $\begingroup$ @nxglogic, what you're describing is robust standard errors, but robust regression does aim to reduce the influence of outliers. An example is MASS::rlm(), asked about here. $\endgroup$
    – Noah
    Commented Apr 12, 2021 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.