I found out that we can calculate some estimator depends on the objective function. Where if we want to minimize the least square $\sum (x_i - \theta)^2$ the best estimator is the mean. And if we want to minimize the absolute difference $\sum |x_i - \theta|$ the best estimator is the median. I was wondering, how about if I want to minimize M-estimator $\sum \rho (x_i - \theta)$, where this objective value usually called as Huber Estimator. How can I compute the estimator for this objective value? Thank you
A solution won't necessarily exist in closed form -- it depends on your choice of $ρ$.
For most situations you can only minimize the loss function iteratively; for a few $\rho$ functions an explicit closed form may exist.
There are a variety of suitable univariate optimization techniques. Any of the standard optimization methods can be used (e.g. the univariate optimizer optimize in R uses Brent's method), but it's fairly common to use iterative reweighted least squares starting from a good robust estimate (often, but not always the median).
In the case of the Huber $M$-estimator a well-chosen trimmed mean may provide an excellent starting point.