I need an advice which portion of a dataset should be used to calculate cuts for discretization. I use two levels of Cross Validation. One is external to the model creating, but the second is used internally by the model learning algorithm.

Current test procedure is as follows:

  1. Use CV5 to split data onto train and test sets

  2. Calculate cuts on train set for continues attributes
  3. Pass discretized train data to the model learning procedure
  4. Apply cuts to test data set
  5. Classify test data using the model

In step 3. internally the learning procedure do as follows:

a. Take the data and split it using CV3

b. Train each model independently using each of the 3 data folds use the rest of the data to measure performance

c. select and return the best model

Is the above procedure correct or should discretization be done by model learning method internally?

  • $\begingroup$ A possibly more important question is: why are you discretizing your continuous variables? $\endgroup$ – Matthew Drury Dec 7 '16 at 16:55
  • $\begingroup$ The model I am testing can only deal with symbolic data $\endgroup$ – Sebastian Widz Dec 7 '16 at 17:11
  • $\begingroup$ Very interesting. $\endgroup$ – Matthew Drury Dec 7 '16 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.