I am reading this post which talks about Robust Linear regression in a Bayesian setting. The particular blog post can be found here:
There was a particular bit which I could not handle. The author talks about robust regression and says that in the presence of outliers a frequentist would use a non-quadratic distance measure to evaluate the fit. Then he or she talks about assuming that the data is distributed according to the student t-distribution with heavier tails.
The bit that has me confused is (speaking about having a normal prior):
As you can see, the fit is quite skewed and we have a fair amount of uncertainty in our estimate as indicated by the wide range of different posterior predictive regression lines. Why is this? The reason is that the normal distribution does not have a lot of mass in the tails and consequently, an outlier will affect the fit strongly.
I do not follow as to why having lighter weight on values far out is making it worst. Any hints as to why that is would be greatly appreciated.