I am quite interested in the field of data analysis and mining and therefore, I have started doing some real problem using SVM regression to predict a target variable. My response variable is expensive and time consuming to measure. Hence, I would like to find a proper function which can predict this variable properly. Therefore, I have found support vector regression as a robust method. But the problem is that I have a couple of explanatory variables which do not have a good correlation with the target variable. Indeed, I can not do a good prediction with them even with SVM regression. Note I have tried several kernels and in addition, I used several tuning methods and even estimate the meta parameter of SVR, but I still have the problem with several outliers.

I mean with outlier is that I have 60 observation which has the value of 3 and 16 as minimum and maximum respectively and then 3 of observations have the value less than 4 and the rest (57) have the value bigger than 4 and less than 16. Therefore, I call the values less than 4 outliers because when I remove them from the observation I have a better prediction accuracy.

In overall, I have two questions which I want to ask:

First, is there any method to improve the relationship between predictors and response? which this leads to a better prediction performance.

Second, is there a way to solve the problem with outlier explained above rather deleting them?

Thanks for taking your time to read my long question.


I have found that my question is a kind of broad. Let me say, Can I use Transformation to increase the relation between target and predictors? If so, How can I back to the real value of target variable(I guess jus inverse computation of that transformation would be right but how)?


1 Answer 1


Since I want to adopt machine learning programs interacting better with the involving component and its accuracy assurance. I am learning about those problems for a couple of machine learning algorithms such as SVM for classification and regression. However, Regarding my question, I have found it to be nice to share what I have learned from my question and answer it with my experience in a particular example.

As I asked I was trying to estimate an expensive variable to measure via using several weak explanatory variables. Therefore, I have employed SVM regression and I have found that transformation has a direct relation with choosing of the kernel. eg. Since I wanted a good approximation, I have transformed all the variables with 2 different transformation method (SQRT and Z score).

after that, I have defined a radial kernel for nu-SVM and ignored the hyperparameter from this kernel and I fitted the model and the result was not satisfactory. therefore, I replaced with the linear kernel for computation and the result was incredibly better than before Transformation and even after using radial (Note: Since the method is linear I have checked the residual distribution and the K fold Cross-validation using RMSE for only Testset/unseen data individually for all 10fold).

In conclusion, As mentioned in lots of literature choosing the kernel is regarding the distribution of the data and therefore, in a wanted case choosing the right transformation has priority before feeding the data into algorithms.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.