Difference between removing outliers and using Least Trimmed Squares? In most cases we would be reluctant to remove outliers from the dataset just to get a better fit. Robust estimators such as Least Trimmed Squares are sometimes recommended in order to fit a regression line without the influence of outliers (or at least weighing them down).
I see that we are keeping the full dataset, so the outlier points will be present in summary statistics, plots, etc. But besides that, is there any other substantial difference between the two approaches? The usual criticisms on not considering datapoints that might be legitimate and correctly reflect the population seem to not be addressed, just circumvented with a formalized method that automates the process.
 A: The reason is largely cultural, in my opinion. Well defined statistical methods are favored in science because they give a transparent analysis of the data. This is probably one of the reason that p-values are so popular.
When an outlier is excluded by a practitioner manually, there may be many factors that might lead to this judgement. A reader of the practitioner's research may need detailed and non-leading explanation before they understand the justification for exclusion of a data point.
In constrast, a method like LTS excludes points based on a clear algorithm. Once the tuning parameters, like the alpha level, are set, it is generally transparent as to why points are excluded. Full disclosure - to some extent the can is being kicked here - there are those selected values for the tuning parameters which still need to be justified. That is similar to the way that the 5% p-value level should be justified.
Besides a an algorithm that can be deconstructed to see why some points are excluded, there are some additional advantages to algorithms. Since substantial work has gone into the development of methods like LTS, some properties about it are already proven (like breakdown value, etc). There is no proof about properties of a person's justification for removing points.
In short, the substantial difference between algorithmic and manual outlier selection exists.
A: Let $(X_i,Y_i),\dots,(X_n,Y_n)$ be an sample. Let $r_i^2(f)=(f(X_i)-Y_i)^2$ Least Trimmed Squared can be written like that:
$$\widehat f= \arg\min_{f \in \mathcal{F}} \sum_{i=1}^k r_{(i)}(f)^2  $$
where the parenthesis means that we sorted the data $r_{(1)}(f)\le \dots\le r_{(n)}(f)$. It is adaptative to the data, we don't threshold at a given value we use the data to know which points are to be excluded and this exclusion depends on $f$ which is not the case when you do outlier removal. Here the outlier removal procedure is kind of embedded in the method and you can't decompose the procedure into two parts outliers removal and then estimation. In some non-complicated cases indeed this would give you the same value but when $\mathcal{F}$ is complicated, when the data are high-dimensional... this is not obvious that you would get the same thing.
Other more involved reasons is that an outlier will not have the same influence (as in influence function, if you are interested you can search this keyword). Suppose we are in a very simple case where $f(x)$ is a constant and call $T(y_1,\dots,y_n)$ the value of $f(x)$ for a given sample $Y_i=y_i$, it means that in fact you are  searching for the mean of the distribution $Y$ and $T(Y_1,\dots,Y_n)$ is a (robust) estimator of the mean. Then, define for $y\in \mathbb{R}$
$$S(y)=|T(Y_1,\dots,Y_n)- T(Y_1,\dots,Y_{n-1},y)| $$
call this the sensitivity of $T$ it correspond to the change of value when changing $Y_n$ for an outlier situated in $y$. For least trimmed square estimator, $S(\infty)$ is not zero if, say $r_{n}(f)=r_{(i)}(f)$ for some $i\le k$.
In a few words, an outlier placed in a very big value will pull the estimator $\widehat f$ towards infinity, not a lot but a little and this means that the outlier has been taken into account and this is not true when using outlier removal techniques in which case you ignore outliers.
