0
$\begingroup$

Throughout my R journey I have noticed the way we can use given data to develop and validate a model.

Assume that you have given data for a problem

  1. train.csv
  2. test.csv

Method A

Combine train+test data and develop a model using the combined data. Then use test.data to validate the model based on predicted error analysis.

Method B

Use train data to develop the model and then use test data to validate the model based on predicted error analysis.

Method C

Sub divided 75% as training data and 25% test data on train.csv file and use new training data for developing the model. Then use new test data to validate the model. After that use initial given test data to double check the performance of the model.

I have identified 3 methods so it is bit confusing which one to use.

Are there any other methods other than these methods?

I need opinions from R experts on

  1. What is the best practice?

  2. Does that depend on the scale of the problem (smaller data or big data)?

  3. a) Confusion matrix is the only way that can we use to check the performance of a model?

    b) Is there any other matrices to check the performance?

    c) Does it depend on the type of the model(lm(),glm(),tree(),svm() etc..)?

    d) Do we have different matrices for different models to evaluate the model?

Improve this question
$\endgroup$
6
  • $\begingroup$ Note that this doesn't have anything to do with R. $\endgroup$
    – gung - Reinstate Monica
    Apr 2, 2016 at 17:38
  • $\begingroup$ @user777 someone told me to post this question on stat page $\endgroup$
    – Harry
    Apr 2, 2016 at 21:29
  • $\begingroup$ @gung did you see the part 3? $\endgroup$
    – Harry
    Apr 2, 2016 at 21:29
  • $\begingroup$ Everything here has been covered on the site already. Please search & read around. The linked thread is the place to start, it already covers most of this. $\endgroup$
    – gung - Reinstate Monica
    Apr 2, 2016 at 23:24
  • $\begingroup$ @C11H17N2O2SNa you should know how to answer these questions and involve community to discuss these types of questions rather than closing it $\endgroup$
    – Harry
    Apr 4, 2016 at 14:24

0

Reset to default