Skip to main content
fixed typos
Source Link
Scortchi
  • 31.6k
  • 9
  • 102
  • 282

As previous posters have mentioned, it is generally best to avoid dichotomizing a continuous variable. However, in answer to your question, there are instances where dichotomizing a continuous variable does confer advantages.

For instance, if a given variable contains missing values for a significant proportion of the population, but is known to be highly predictive and the missing values themselves bear predictive value. For example, in a credit scoring model, consider a variable, let's say average-revolving-credit-balance (which granted, is not technically continuous, but in this case mirrors a normal distribution close enough to be treated as such), which contains missing values for about 20% of the applicant pool in a given target market. In this case, the missing values for this variable represent a distinct class--those who don't have an open, revolving-credit line; these customers will display entirely different behavior compared to, say, those with available revolving credit-lines, but whomwho regularly carry no balance. If instead these missing values were discarded, or imputed, it could restrict the model's predictive ability.

Another benefit of dichotimizationdichotomization: it can be used to mitigate the effects of significant outliers that skew coefficients, but represent realistic cases that need to be handled. If the outliers don't differ greatly in outcome from other values in the nearest percentiles, but skew the parameters enough to effect marginal accuracy, then it may be beneficial to group them with values displaying similar effects.

Sometimes a distribution naturally lends itself to a set of classes, in which case dichotimizationdichotomization will actually give you a higher degree of accuracy than a continuous function.

Also, as previously mentioned, depending on the audience, the ease of presentation can outweigh the losses to accuracy. To use credit scoring again as an example, in practice, the high-degree degree of regulation does make a practical case for discretizing at times. While the higher degree of accuracy could help the lender cut losses, practitioners must also consider that models need to be easily understood by regulators (who may request thousands of pages of model documentation) and consumers, whom if denied credit, are legally entitled to an explanation of why.

It all depends on the problem at hand and the data, but there are certainly cases where dichotimizationdichotomization has its merits.

As previous posters have mentioned, it is generally best to avoid dichotomizing a continuous variable. However, in answer to your question, there are instances where dichotomizing a continuous variable does confer advantages.

For instance, if a given variable contains missing values for a significant proportion of the population, but is known to be highly predictive and the missing values themselves bear predictive value. For example, in a credit scoring model, consider a variable, let's say average-revolving-credit-balance (which granted, is not technically continuous, but in this case mirrors a normal distribution close enough to be treated as such), which contains missing values for about 20% of the applicant pool in a given target market. In this case, the missing values for this variable represent a distinct class--those who don't have an open, revolving-credit line; these customers will display entirely different behavior compared to, say, those with available revolving credit-lines, but whom regularly carry no balance. If instead these missing values were discarded, or imputed, it could restrict the model's predictive ability.

Another benefit of dichotimization: it can be used to mitigate the effects of significant outliers that skew coefficients, but represent realistic cases that need to be handled. If the outliers don't differ greatly in outcome from other values in the nearest percentiles, but skew the parameters enough to effect marginal accuracy, then it may be beneficial to group them with values displaying similar effects.

Sometimes a distribution naturally lends itself to a set of classes, in which case dichotimization will actually give you a higher degree of accuracy than a continuous function.

Also, as previously mentioned, depending on the audience, the ease of presentation can outweigh the losses to accuracy. To use credit scoring again as an example, in practice, the high-degree of regulation does make a practical case for discretizing at times. While the higher degree of accuracy could help the lender cut losses, practitioners must also consider that models need to be easily understood by regulators (who may request thousands of pages of model documentation) and consumers, whom if denied credit, are legally entitled to an explanation of why.

It all depends on the problem at hand and the data, but there are certainly cases where dichotimization has its merits.

As previous posters have mentioned, it is generally best to avoid dichotomizing a continuous variable. However, in answer to your question, there are instances where dichotomizing a continuous variable does confer advantages.

For instance, if a given variable contains missing values for a significant proportion of the population, but is known to be highly predictive and the missing values themselves bear predictive value. For example, in a credit scoring model, consider a variable, let's say average-revolving-credit-balance (which granted, is not technically continuous, but in this case mirrors a normal distribution close enough to be treated as such), which contains missing values for about 20% of the applicant pool in a given target market. In this case, the missing values for this variable represent a distinct class--those who don't have an open, revolving-credit line; these customers will display entirely different behavior compared to, say, those with available revolving credit-lines, but who regularly carry no balance. If instead these missing values were discarded, or imputed, it could restrict the model's predictive ability.

Another benefit of dichotomization: it can be used to mitigate the effects of significant outliers that skew coefficients, but represent realistic cases that need to be handled. If the outliers don't differ greatly in outcome from other values in the nearest percentiles, but skew the parameters enough to effect marginal accuracy, then it may be beneficial to group them with values displaying similar effects.

Sometimes a distribution naturally lends itself to a set of classes, in which case dichotomization will actually give you a higher degree of accuracy than a continuous function.

Also, as previously mentioned, depending on the audience, the ease of presentation can outweigh the losses to accuracy. To use credit scoring again as an example, in practice, the high degree of regulation does make a practical case for discretizing at times. While the higher degree of accuracy could help the lender cut losses, practitioners must also consider that models need to be easily understood by regulators (who may request thousands of pages of model documentation) and consumers, whom if denied credit, are legally entitled to an explanation of why.

It all depends on the problem at hand and the data, but there are certainly cases where dichotomization has its merits.

Source Link

As previous posters have mentioned, it is generally best to avoid dichotomizing a continuous variable. However, in answer to your question, there are instances where dichotomizing a continuous variable does confer advantages.

For instance, if a given variable contains missing values for a significant proportion of the population, but is known to be highly predictive and the missing values themselves bear predictive value. For example, in a credit scoring model, consider a variable, let's say average-revolving-credit-balance (which granted, is not technically continuous, but in this case mirrors a normal distribution close enough to be treated as such), which contains missing values for about 20% of the applicant pool in a given target market. In this case, the missing values for this variable represent a distinct class--those who don't have an open, revolving-credit line; these customers will display entirely different behavior compared to, say, those with available revolving credit-lines, but whom regularly carry no balance. If instead these missing values were discarded, or imputed, it could restrict the model's predictive ability.

Another benefit of dichotimization: it can be used to mitigate the effects of significant outliers that skew coefficients, but represent realistic cases that need to be handled. If the outliers don't differ greatly in outcome from other values in the nearest percentiles, but skew the parameters enough to effect marginal accuracy, then it may be beneficial to group them with values displaying similar effects.

Sometimes a distribution naturally lends itself to a set of classes, in which case dichotimization will actually give you a higher degree of accuracy than a continuous function.

Also, as previously mentioned, depending on the audience, the ease of presentation can outweigh the losses to accuracy. To use credit scoring again as an example, in practice, the high-degree of regulation does make a practical case for discretizing at times. While the higher degree of accuracy could help the lender cut losses, practitioners must also consider that models need to be easily understood by regulators (who may request thousands of pages of model documentation) and consumers, whom if denied credit, are legally entitled to an explanation of why.

It all depends on the problem at hand and the data, but there are certainly cases where dichotimization has its merits.