My continuous dependent variable has a lot of error in it. Hence, I was thinking of discretizing it, to reduce the error for my modeling effort. But firstly, the main focus of my modeling effort are the following: 1) Determine whether there is any predictive power in the set of independent variables (IVs) that I have; 2) If (1) is true, ie the IVs have predictive power, I'd like to do feature selection: I have ~40 IVs, many of them are correlated. I would like to select a parsimonious set (4 or 5 IVs) that predict the DV to some level.

In this context, I found this para from the "Applied predictive modeling" (by Max Kuhn & Kjell Johnson, 2013) to be relevant:

"A second common reason for wanting to categorize a continuous response is that the scientist may believe that the continuous response contains a high degree of error, so much so that only the response values in either extreme of the distribution are likely to be correctly categorized. If this is the case, then the data can be partitioned into three categories, where data in either extreme are classified generically as positive and negative, while the data in the midrange are classified as unknown or indeterminate. The middle category can be included as such in a model (or specifically excluded from the model tuning process) to help the model more easily discriminant between the two categories."

(See section 20.4, "Discretizing Continuous Outcomes", at the end; pg 533).

In my case, I think that the strategy outlined in the boldface part above (where the middle category has been excluded from the model tuning process) makes sense. That way, we are defining two well-separated classes. These well-separated classes help me "analyze the set of IVs" by using a model (so to speak) and do feature selection. By the way, I am planning to use glmnet/lasso for the feature selection.

My question to you is: Can you please give me two to three good references (ie, journal papers) where dependent variable discretization has been justified in this way?

Thanks in advance! R.

My continuous dependent variable has a lot of error in it. Hence, I was thinking of discretizing it, to reduce the error for my modeling effort. But firstly, the main focus of my modeling effort are the following: 1) Determine whether there is any predictive power in the set of independent variables (IVs) that I have; 2) If (1) is true, ie the IVs have predictive power, I'd like to do feature selection: I have ~40 IVs, many of them are correlated (num. of observations is 150). I would like to select a parsimonious set (4 or 5 IVs) that predict the DV to some level.

In this context, I found this para from the "Applied predictive modeling" (by Max Kuhn & Kjell Johnson, 2013) to be relevant:

"A second common reason for wanting to categorize a continuous response is that the scientist may believe that the continuous response contains a high degree of error, so much so that only the response values in either extreme of the distribution are likely to be correctly categorized. If this is the case, then the data can be partitioned into three categories, where data in either extreme are classified generically as positive and negative, while the data in the midrange are classified as unknown or indeterminate. The middle category can be included as such in a model (or specifically excluded from the model tuning process) to help the model more easily discriminant between the two categories."

(See section 20.4, "Discretizing Continuous Outcomes", at the end; pg 533).

In my case, I think that the strategy outlined in the boldface part above (where the middle category has been excluded from the model tuning process) makes sense. That way, we are defining two well-separated classes. These well-separated classes help me "analyze the set of IVs" by using a model (so to speak) and do feature selection. By the way, I am planning to use glmnet/lasso for the feature selection.

My question to you is: Can you please give me two to three good references (ie, journal papers) where dependent variable discretization has been justified in this way?

Thanks in advance! R.

My continuous dependent variable has a lot of error in it. Hence, I was thinking of discretizing it, to reduce the error for my modeling effort. Hence, I found this para from the "Applied predictive modeling" (by Max Kuhn & Kjell Johnson, 2013) to be relevant:

"A second common reason for wanting to categorize a continuous response is that the scientist may believe that the continuous response contains a high degree of error, so much so that only the response values in either extreme of the distribution are likely to be correctly categorized. If this is the case, then the data can be partitioned into three categories, where data in either extreme are classified generically as positive and negative, while the data in the midrange are classified as unknown or indeterminate. The middle category can be included as such in a model (or specifically excluded from the model tuning process) to help the model more easily discriminant between the two categories."

(See section 20.4, "Discretizing Continuous Outcomes", at the end; pg 533).

My question to you is: Can you please give me two to three good references (ie, journal papers) where dependent variable discretization has been justified in this way? Specifically, I am interested in the boldface part above: where the middle category has been excluded from the model tuning process.

Thanks in advance! R.

My continuous dependent variable has a lot of error in it. Hence, I was thinking of discretizing it, to reduce the error for my modeling effort. But firstly, the main focus of my modeling effort are the following: 1) Determine whether there is any predictive power in the set of independent variables (IVs) that I have; 2) If (1) is true, ie the IVs have predictive power, I'd like to do feature selection: I have ~40 IVs, many of them are correlated. I would like to select a parsimonious set (4 or 5 IVs) that predict the DV to some level.

In this context, I found this para from the "Applied predictive modeling" (by Max Kuhn & Kjell Johnson, 2013) to be relevant:

"A second common reason for wanting to categorize a continuous response is that the scientist may believe that the continuous response contains a high degree of error, so much so that only the response values in either extreme of the distribution are likely to be correctly categorized. If this is the case, then the data can be partitioned into three categories, where data in either extreme are classified generically as positive and negative, while the data in the midrange are classified as unknown or indeterminate. The middle category can be included as such in a model (or specifically excluded from the model tuning process) to help the model more easily discriminant between the two categories."

(See section 20.4, "Discretizing Continuous Outcomes", at the end; pg 533).

In my case, I think that the strategy outlined in the boldface part above (where the middle category has been excluded from the model tuning process) makes sense. That way, we are defining two well-separated classes. These well-separated classes help me "analyze the set of IVs" by using a model (so to speak) and do feature selection. By the way, I am planning to use glmnet/lasso for the feature selection.

My question to you is: Can you please give me two to three good references (ie, journal papers) where dependent variable discretization has been justified in this way?

Thanks in advance! R.