Finding the right model I have a data frame with several predictors, lets call them pred1 through pred3, and a result column. Now I need to specify the right model. I could randomly try:
svm.model <- svm(result ~ pred1+pred2+pred3,       data = train)
# or
svm.model <- svm(result ~ pred1*pred2+pred3^2,     data = train)
# or
svm.model <- svm(result ~ log2(pred1)+pred2*pred3, data = train)
# etc. etc.

But there must be a better automatic approach in R to model selection?!
 A: For unknown nonlinear dependences, there are nonlinear models. The most commonly used are tree models, i. e. random forest (package randomForest) and boosted trees (package gbm) and neural networks.
If you want to use linear regression for nonlinear dependence, one possibility is to create a big polynom with lots of terms, and then do stepwise elimination.  Use logarithms if (and only if) the feature change orders of magnitude, i.e. you try to predict the life span of arbitrary animal from its weight.
Another possibility is to try Kernel Ridge Regression.
You may also want to read about Generalized Additive Model.
A: I would be very cautious in applying these "random" methods. Any model has to have a strong background in discipline-specific theory. 
If you do find a satisfactory model using the algorithms suggested, you're still left with the formidable task of justifying why it theoretically makes sense. "The stats model says that pred1 and pred3 predict the result" is not an acceptable answer. 
With a different run of the pred1, pred2, pred3, the relationship you thought you built may prove no longer hold. In that case, the whole exercise becomes invalid.
