A universal measure of the accuracy of linear regression models I have a dataset that contains both outliers and multicollinearity.
I applied three different regression models to that dataset: ordinary least square, absolute linear regression, and Huber regression.
My goal is to test which linear regression model is good or accurate at dealing with combined outliers and multicollinearity issues in a dataset.
I cannot use mean square error or R^2 because ordinary square regression will be the best, and the same with median absolute deviation, since absolute linear regression will be the best.
Which measures can I use to judge which model is more accurate than others?
Clarification 
The dataset has been split into train and test data, where the regression model is applied to the train dataset, and the performance measure is calculated using an unseen dataset (the test set). 
I want to check how accurate the model is in estimating or predicting the coefficients.
In other words, do the outliers and multicollinearity in a dataset affect the linear regression model and thus shift the coefficients too much? 
I want a universal measure that I can judge and say, with confidence, that this linear regression model is good compared to others, even in the presence of outliers and multicollinearity simultaneously. 
UPDATE
My question, in simple words, is: if I have a dataset and this dataset contains outliers and multicollinearity, then I applied three different regression models to that dataset. I want to investigate the accuracy of these regression models based on a performance measure that is not biased towards any particular regression model. After calculating this measure, I can say with confidence that this regression model is the best one even when the dataset contains outliers and multicollinearity.
NOTE
I say one dataset solely for the sake of simplification. Over 30 databases and a hundred simulations are included in the project. Additionally, it includes more than ten regression models.
New Update after Dave answer
When I analyse this real dataset, the case is quite different from what Dave described. When MSE is used as the performance metric, OLS is the optimal choice; when MAD is used, linear absolute regression is the superior choice.
rm(list=ls())
library(L1pack) 
library(glmnet)
library(MASS)
library(robust)
library(robustbase)
library(quantreg)
library(readr)
library(readxl)
####################################### 
## mean of the k-fold results 
mee=function(x){
mmm=rep(0,ncol(x))
for (i in 1:ncol(x)){
mmm[i]=mean(x[,i])
}
return(mmm)
}
###############  Dataset  ################
wdbc <- read_excel("Folds5x2_pp.xlsx") 
###############################
x=as.matrix(wdbc[,-5])
y=as.vector(wdbc[,5])
wdbc<-as.data.frame(cbind(x,y))  
n=nrow(wdbc)
################################
# K-fold Cross-validation
k=30 # number of folds 
        
folds <- cut(seq(1,n),breaks=k,labels=FALSE)
mols1=matrix(0,nrow= k);mM1=matrix(0,nrow= k);mMM1=matrix(0,nrow= k)
## split the data to train and test set
for(i in 1:k){
testIndexes <- which(folds==i,arr.ind=TRUE)
testData <- wdbc[testIndexes, ]
trainData <- wdbc[-testIndexes, ]
xtr=as.matrix(trainData[,-5])
ytr=trainData[,5]
xte=as.matrix(testData[,-5])
yte=testData[,5]
        
mest=rlm(ytr~xtr,psi=psi.huber,maxit=300)$coefficients
mmest=rq(ytr~xtr, tau = 0.5)$coefficients
ols=lm(ytr~xtr)$coefficients
        
# MSE measure 
mols1[i]=mean((yte-cbind(1,xte)%*%ols)^2)
mM1[i]=mean((yte-cbind(1,xte)%*%mest)^2)
mMM1[i]=mean((yte-cbind(1,xte)%*%mmest)^2)
          
## Use this for MAD measure
## mols1[i]=mean(abs(yte-cbind(1,xte)%*%ols))
## mM1[i]=mean(abs(yte-cbind(1,xte)%*%mest))
## mMM1[i]=mean(abs(yte-cbind(1,xte)%*%mmest))
    
}
res2=cbind(mols1,mM1,mMM1)
MEE=mee(res2)
nam=c("OLS","Huber","Absolute Linear regression")
ty=data.frame(nam,MEE,rank(MEE))
View(ty)

    

 A: RMSE tells us how far the model residuals are from zero on average, i.e. the average distance between the observed values and the predicate values. However, Willmott et. al. suggested that RMSE might be misleading to assess the model performance since RMSE is a function of the average error and  the distribution of squared errors. Chai recommended to use both RMSE and mean absolute error (MAE). It is better to report both metrics. By the way, $R^2$ is misleading as well since it increases for higher number of predictors. I would recommend to use adjusted $R^2$. This metric is kinda gold standard goodness of fit test. To handle multicollinearity, compute the nonparametric Spearman's rank-order correlation and drop the variable close to 1. It will solve the problem. For the outliers, it is not a good practice to delete outliers because they may have valuable insights. Find the influential outliers and fit the model with and without them to see their impact on the model. Also, don't forget to check all four properties of linear model assumptions. Attached articles will give u more explanations.
Reference:
http://www.jstor.org/stable/24869236
https://gmd.copernicus.org/articles/7/1247/2014/
