> There is quite some number of ways how to robustly fit a linear regression model, e.g. using M-estimation based on Tukey's biweight loss or on Huber's loss, see e.g. Wikipedia.

> Is it correct that these are modelling the mean of the response as long as the error distribution is symmetric?

Well, estimating, at least -- but *only* if the mean exists. But they'll be reasonable estimates of the center of symmetry more generally. For example, consider the $t$ distribution with $\nu$ degrees of freedom. If $\nu\leq 1$ there's no mean.

> What location parameter are they modelling in general? 

M-estimation corresponds to maximum-likelihood estimation if the loss function corresponds to $-\log(f)$  for some density $f$. This is the case for the Huber loss but not the Tukey.

> Is it a specially weighted average of the response variable?

Not in general no\*. But M-estimators can be obtained\*\* by iterating a weighted average where the weights are updated at each step.

\*(at least not unless 'special' is interpreted much more broadly than people generally understand the term 'weighted average')

\*\*(in some circumstances at least)