How to Improve the relationship between predictors and observation for having a better fit I am quite interested in the field of data analysis and mining and therefore, I have started doing some real problem using SVM regression to predict a target variable. My response variable is expensive and time consuming to measure. Hence, I would like to find a proper function which can predict this variable properly. Therefore, I have found support vector regression as a robust method. But the problem is that I have a couple of explanatory variables which do not have a good correlation with the target variable. Indeed, I can not do a good prediction with them even with SVM regression. Note I have tried several kernels and in addition, I used several tuning methods and even estimate the meta parameter of SVR,  but I still have the problem with several outliers. 
I mean with outlier is that I have 60 observation which has the value of 3 and 16 as minimum and maximum respectively and then 3 of observations have the value less than 4 and the rest (57) have the value bigger than 4 and less than 16. Therefore, I call the values less than 4 outliers because when I remove them from the observation I have a better prediction accuracy.
In overall, I have two questions which I want to ask:
First, is there any method to improve the relationship between predictors and response? which this leads to a better prediction performance.
Second, is there a way to solve the problem with outlier explained above rather deleting them?
Thanks for taking your time to read my long question.
Edited 
I have found that my question is a kind of broad. Let me say, Can I use Transformation to increase the relation between target and predictors? If so, How can I back to the real value of target variable(I guess jus inverse computation of that transformation would be right but how)? 
 A: Since I want to adopt machine learning programs interacting better with the involving component and its accuracy assurance. I am learning about those problems for a couple of machine learning algorithms such as SVM for classification and regression. However, Regarding my question, I have found it to be nice to share what I have learned from my question and answer it with my experience in a particular example.
As I asked I was trying to estimate an expensive variable to measure via using several weak explanatory variables. Therefore, I have employed SVM regression and I have found that transformation has a direct relation with choosing of the kernel. eg. Since I wanted a good approximation, I have transformed all the variables with 2 different transformation method (SQRT and Z score). 
after that, I have defined a radial kernel for nu-SVM and ignored the hyperparameter from this kernel and I fitted the model and the result was not satisfactory. therefore, I replaced with the linear kernel for computation and the result was incredibly better than before Transformation and even after using radial (Note: Since the method is linear I have checked the residual distribution and the K fold Cross-validation using RMSE for only Testset/unseen data individually for all 10fold). 
In conclusion, As mentioned in lots of literature choosing the kernel is regarding the distribution of the data and therefore, in a wanted case choosing the right transformation has priority before feeding the data into algorithms.  
