0
$\begingroup$

Context

If a level within a categorical variable has less observations than there are elements in the training set then there's a chance that all of the elements of that level will be contained within the training set.

If a level has less observations than there are elements in the test set then there's a chance that they'll all be in either test or training.

Clearly the fewer elements there are the higher the chance of them being in just one of the sets, which is problematic for prediction.

I'm interested in how this is typically handled, and whether variables for which this is a possibility are typically discarded, or if there's an acceptable level of probability for this occurring.

Example

Number of rows in data = 559. Split this into $70\%$ training and $30\%$ testing.

So training has 391 rows, testing has 168 rows.

  • prediction variable : direction
  • levels : BL BM BR ML MM MR TL TM TR (where BR=bottom right, ML=middle left,TM=top middle etc)
  • response variable : event
  • levels : 0, 1

There were only 5 observations for any of the TR,TM,TL. Giving a good chance that all of them would be contained in either the test or the training.

I'm wondering what's suggested for handling this.

Example continued

Here's the variable for the trainset, testest.

> table(trainset$POSITION)

BL  BM  BR  ML  MM  MR  TL  TR 
177   8 138  25  16  22   3   2 
> table(testset$POSITION)

BL BM BR ML MM MR TL TR 
68  3 63 18  7  9  0  0 

Merging these into top,middle,bottom doesn't help

> table(testset$topmiddlebottom)

  B   M   T 
134  34   0 
> table(trainset$topmiddlebottom)

  B   M   T 
323  63   5 

Combining these into bottom, not bottom gives the following (where NB=not bottom)

  B  NB 
457 102 

As there are 102 values it's still possible for them to be entirely containing within either testing or training.

Is there some kind of minimal number of values to use for prediction? Is it required that there are at least as many values in the level as there are elements in the smallest set (so that's elements in testing set).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.