0
$\begingroup$

I am R-tool beginner. I have a question regarding how to know the performance of a linear regression model by using validation data. My approach was

  1. Create training and validation data sets from original data set. "train" is name of my training data set and "valid" is name of my validation data set. "category" will be my target variable and "date_time" is my independent variable.

  2. Use training data set to create a regression model

    attach(train)

    lreg=lm(category~date_time)

  3. Now do predictions for validation data set using model created with training data set

    p=predict(lreg,valid)

  4. Now check the accuracy by finding the values of ACC, AUC.

    mmetric(valid$category,p,"AUC")

    mmetric(valid$category,p,"ACC")

Now if AUC and ACC have small values then it means that model created by training data set is not good in making predictions.

Is my approach correct ?

Thanks and regards!

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Your questions are nearly entirely addressed at stats.stackexchange.com/questions/43310/… . $\endgroup$
    – rolando2
    Commented Nov 16, 2012 at 0:12
  • $\begingroup$ Thanks for redirecting me. As per my understanding I think I am doing the correct thing in my question. creating a model with training data and doing prediction of validation data. Then calculating the performance. But I shall really appreciate it if someone could just answer in Yes or No so that I can be more certain. Regards $\endgroup$
    – user16603
    Commented Nov 16, 2012 at 22:02
  • 2
    $\begingroup$ Please do not erase your question and replace it w/ something else. CV is intended to build a permanent repository of information about statistics via archived questions & answers. You are welcome to edit your Q and add updated info below the original question, but don't just erase what you initially asked. You may want to read our FAQ. $\endgroup$
    – gung - Reinstate Monica
    Commented Dec 17, 2012 at 0:47
  • $\begingroup$ You might want to explain what your outcome is like. For example, are the 9 categories ordered? $\endgroup$
    – dimitriy
    Commented Jan 16, 2013 at 1:48

1 Answer 1

Reset to default
1
$\begingroup$

You should take such a result w/a pinch of salt. For instance, due to a particular mix, despite a bad train result, you can get a good test result. Therefore, it's better if you do a cross-validation-like approach and try to define multiple train-test scenario and see if you have consistency across them.

PS, if your output is like classes, you may wanna you classification, not regression ...

Improve this answer
$\endgroup$
1
  • $\begingroup$ thanks, Categories have values from 1 -9. And my aim was to guess category from independent variables. So I am confused about what to do. Should I do classification or regression? Can you please tell bit about how to perform classification in r-tool? However also please have a look at my question here: stats.stackexchange.com/questions/43040/… $\endgroup$
    – user16603
    Commented Nov 15, 2012 at 22:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.