Skip to main content
Cross Validated

Return to Question

edited tags
Link
kjetil b halvorsen ♦
kjetil b halvorsen ♦
  • 82.8k
  • 32
  • 201
  • 663
Tweeted twitter.com/#!/StackStats/status/236748316241719296
deleted 1 characters in body; edited tags
Source Link
chl
chl
  • 54.3k
  • 23
  • 227
  • 388

I have a huge dataset (n$n$ around five million, p$p$ around three thousand) for a classification problem, where my interest is predictive class probabilities for test data, not the target. I shall be using bootstrap samples to estimate the smoothed probabilities from various models.

The problem is when I bootstrap the cases not all the levels in the categorical predictors will be in every resample. Given that I need parameters for all levels when working with the test set, I'm unsure of what to do.

I read Agresti's Categorical data modellingCategorical Data Analysis, but there doesn't seem to be mention of this.

I have thought of 2 possibilities:

  1. Insert a base composition of varied cases in every resample so that all levels are included for all predictors.

  2. Define all levels for each categorical variable with reference to the data schema and run the models.

I'm having to think about this because my resamples are pulled as csvCSV data from a sqlSQL database and I would usually use read.csv()read.csv() which automagically handles levels, preference relations and levels for categorical data using the cases in the csvCSV file. (Pulling all of the data in one csv is not an option due to resource constraints.)

Thanks

I have a huge dataset (n around five million, p around three thousand) for a classification problem, where my interest is predictive class probabilities for test data, not the target. I shall be using bootstrap samples to estimate the smoothed probabilities from various models.

The problem is when I bootstrap the cases not all the levels in the categorical predictors will be in every resample. Given that I need parameters for all levels when working with the test set, I'm unsure of what to do.

I read Agresti's Categorical data modelling, but there doesn't seem to be mention of this.

I have thought of 2 possibilities:

  1. Insert a base composition of varied cases in every resample so that all levels are included for all predictors.

  2. Define all levels for each categorical variable with reference to the data schema and run the models.

I'm having to think about this because my resamples are pulled as csv data from a sql database and I would usually use read.csv() which automagically handles levels, preference relations and levels for categorical data using the cases in the csv file. (Pulling all of the data in one csv is not an option due to resource constraints.)

Thanks

I have a huge dataset ($n$ around five million, $p$ around three thousand) for a classification problem, where my interest is predictive class probabilities for test data, not the target. I shall be using bootstrap samples to estimate the smoothed probabilities from various models.

The problem is when I bootstrap the cases not all the levels in the categorical predictors will be in every resample. Given that I need parameters for all levels when working with the test set, I'm unsure of what to do.

I read Agresti's Categorical Data Analysis, but there doesn't seem to be mention of this.

I have thought of 2 possibilities:

  1. Insert a base composition of varied cases in every resample so that all levels are included for all predictors.

  2. Define all levels for each categorical variable with reference to the data schema and run the models.

I'm having to think about this because my resamples are pulled as CSV data from a SQL database and I would usually use read.csv() which automagically handles levels, preference relations and levels for categorical data using the cases in the CSV file. (Pulling all of the data in one csv is not an option due to resource constraints.)

Source Link
Yoda
Yoda
  • 379
  • 1
  • 9

Absent categorical data levels in Bootstrap samples

I have a huge dataset (n around five million, p around three thousand) for a classification problem, where my interest is predictive class probabilities for test data, not the target. I shall be using bootstrap samples to estimate the smoothed probabilities from various models.

The problem is when I bootstrap the cases not all the levels in the categorical predictors will be in every resample. Given that I need parameters for all levels when working with the test set, I'm unsure of what to do.

I read Agresti's Categorical data modelling, but there doesn't seem to be mention of this.

I have thought of 2 possibilities:

  1. Insert a base composition of varied cases in every resample so that all levels are included for all predictors.

  2. Define all levels for each categorical variable with reference to the data schema and run the models.

I'm having to think about this because my resamples are pulled as csv data from a sql database and I would usually use read.csv() which automagically handles levels, preference relations and levels for categorical data using the cases in the csv file. (Pulling all of the data in one csv is not an option due to resource constraints.)

Thanks