Loss function of linear regression How do we decide whether mean absolute error or mean square error is better for linear regression? Are there other loss functions that are commonly used for linear regression?
 A: Put simply: it matters what error metric matters most to you. I have not personally seen a useful application of absolute error loss. In 99% of cases, people use squared error loss. Regression, by definition, is about modeling trend lines that approximate a mean response over a range of predictors. 
If the CLT applies, that mean response tends to a normal distribution. Squared error loss minimax estimator of the mean gives the MLE for normally distributed data. This means that asymptotically, you will get correct 95% CI and p-values for the statistical tests about model parameters.
Even the small sample performance of squared error loss is surprising and favorable, based on my experience. Robust error estimation by bootstrap is usually not profoundly different, or when it is, it seems to be a result of small sample sizes in which both methods perform poorly ("No Free Lunch" theorem for statistics). 
In a somewhat pathological alternate example, if the errors are double exponential, the absolute error minimax estimator of the mean gives the MLE. There are some strange sorts of counterexamples to the CLT where the asymptotic distribution of the test statistic tends toward exponential (Huzurbazar), and thus with a mixture could be double exponential.
A concluding remark is one of efficiency. In the case of mean estimation: the central limit theorem says the rate of convergence to the limiting distribution is root N. This is because $\sqrt{n} \left( \hat{\theta}_{L_2} - \theta \right) \rightarrow_d $ a non-singular distribution. As @whuber points out in the comments below, the absolute error loss yields the median as the optimal measure of central tendency in a univariate estimation case. Like the mean, the median also tends to a normal distribution as $n \rightarrow \infty$ but at a much lower rate. In fact, $n^{1/4} \left( \hat{\theta}_{L_1} - \theta \right) \rightarrow d $ a non-singular distribution. So we say that the median is a root-root-n consistent estimator whereas the mean is a root-n consistent estimator: the mean goes to what it's estimating much faster, and thus provides more powerful tests.
Hence, if a solution is to be offered on strictly statistical terms, I would prefer squared error loss not just for it's predominant usage, but for it's theoretically sound probability model for the residuals given correct mean model specification.
