# re-fitting linear regression after data cleaning

I don't have a good knowledge of statistic but i have looked for 3 hours without resolving the problem so i think to ask

I have two variables and i have made a linear regression my question is: if i would like to remove same values to have a more fitting line how should i proceed?

i tried removing high values of mean, difference and sd but as i later discovered i was doing it wrong

• Why do you want to remove values to improve the fit of the line? If that's your data, and that's the line that your regression has fitted to the data, then what is your motivation for trying to change it? – Ian_Fin Jul 13 '16 at 10:24
• we are doing a study about a comparison between self-reported (questionnaire, subjective) and objective measurement of physical activity and to show a preview of the study before going to check all the answer in the questionnaire they asked me to hide values too far from the regression – aster94 Jul 13 '16 at 10:28
• While it may be nicer looking, it's also misleading to present a regression line fitted to a full data set, alongside a set of data points subsequently handpicked to be closer to that regression line. If there are data points there that reflect outliers then it may be worth removing them, following a defensible approach, but then you'd refit the regression it. – Ian_Fin Jul 13 '16 at 10:44
• Yeah, that's possible. You should read the material IcannotFixThis has suggested. If that is the case though then you really should reconsider removing those items. That nice looking graph will be statistically dishonest. – Ian_Fin Jul 13 '16 at 11:14
• Whatever you do, take a principled approach. If you try multiple ways of messing with the data until it looks nice, your conclusions would be misleading and invalid. Any further statistics you run would be over-optimistic because they wouldn't take into account the earlier procedures that were done explicitly to get a good fit. en.wikipedia.org/wiki/Data_dredging – user20160 Jul 13 '16 at 12:21

I guess, sometimes you even Need to learn how to lie with statistics. I am not going to comment or judge. The easiest way would be to look at the residuals and kill the point with the largest residual before repeatind fitting and checking residuals. The plot looks as if it was made with R, so here is a hint to R: The linear regression is done using lm() and that Returns an object with a slot "residuals". The number of the point with the largest residual can be found with which.max:

x<-rnorm(100)
y <- 2*x+5+rnorm(100)

which.max(lm(y~x)\$residuals)

• I agree with the spirit of the answer, but this approach would only remove one data point. The question did talk about removing values, plural. Perhaps there should be an iterative process? – Ian_Fin Jul 13 '16 at 13:09
• @Ian_Fin I had an iterative process in mind, when I wrote "before repeatind fitting and checking residuals". Once one point is removed, all the residuals are not correct anymore and I propose to compute them anew before proceeding. Thanks for the clarification! – Bernhard Jul 13 '16 at 13:12
• Well, @Bernhard, your comment above is not 100% correct - I fear you are confusing residuals and "influence". High residual points might or might not have big influence – IcannotFixThis Jul 13 '16 at 19:55
• @IcannotFixThis Fear not! First: High residual points might have big influence and therefore my recommendation stands, to remove one at a time and calculate residuals every time anew. If it was not influencial, than little is lost. Second: the OP wants to improve the visual impression of his regression line. An influential point may very well be situated very close to the regression line and therefore it seems logical, not to remove it just because it is influential. I must admit though, that I am neither trained nor experienced in massaging data into linear form to make for a nice plot. – Bernhard Jul 13 '16 at 20:35

Without getting in the dispute concerning whether removing points from your data set might or might not make sense, I think it might be useful clarify the difference between outliers, leverage and influence

I believe such framework would help you understand the role played by each point to your regression.