0
$\begingroup$

I have a numeric dataset involving 34 input variables and one output variable.

I'd like to fit a model to the data so to be able to make predictions (regression problem).

So far I've tested MARS (Multivariate Adaptive Regression Spline, earth package), simple random forest (Random Forest, randomForest package), neural networks (Model Averaged Neural Network, package nnet), support vector machines (kernlab package, polynomial, RBF, laplace kernels). I'm using R.

The best model (both in train and test data sets) is the Laplace kernel (R2 = 89%) but the SVR polynomial (degree = 2), the RBF and the neural networks are close seconds (with 87-88%) and the random forest with 85% and MARS with 82%.

Do you suggest any other model that I could try to fit to my data set, based on the results I've fgot so far and on the type (numeric) and size (700 values for each variable) of the data set? Thanks

link to data

I'd like to predict R (last column of data set)

| cite | improve this question | |
$\endgroup$
  • 1
    $\begingroup$ This seems kind of broad and unfocused; can you make your question more specific? $\endgroup$ – Glen_b Sep 5 '14 at 2:31
  • $\begingroup$ I rephrased my question. Basically as I'm not an experiencee user in regression models I'm asking for more experienced voices that could tell me for example: "hey for that data type I'd try YYY model as well". $\endgroup$ – jpcgandre Sep 5 '14 at 2:37
  • 1
    $\begingroup$ linear regression, regression with regularization, regression with restricted cubic splines, Cubist, Boosted tree... $\endgroup$ – charles Sep 5 '14 at 2:50
  • $\begingroup$ @jpcgandre that could be a lot of work! might just stick with boosted trees since your interest seems to be with machine learning. Cubist has limited literature base. Linear regression should be there if you're comparing models - but doesn't look to be where your interest lies. $\endgroup$ – charles Sep 5 '14 at 3:05
  • $\begingroup$ Yes, indeed I was thinking in checking the last two models. Thanks for the tip! $\endgroup$ – jpcgandre Sep 5 '14 at 3:10
1
$\begingroup$

Models you should consider, depends on the type of the response variable (Qualitative or Quantitative).

(Assuming your response is quantitative) Other than the ones you've already used, you may try out almost any regression model (Linear regression, Lasso regression, Kernel regression, Gaussian process regression, Linear least Square regression, and so on, except Logistic regression as it is for categorical responses).

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

You might want to read this question I asked on Data Science before you start going crazy with the models: https://datascience.stackexchange.com/questions/992/why-might-several-types-of-models-give-almost-identical-results

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.