How to reduce the number of labels in regression I'm working in a regression problem, related to bio-signals, where my labels are integer numbers between 0 and 10.
I've tried a couple of regression algorithms already, mainly linear regression.
Edit: More precisely my dataset is composed by 105 rows, and I have 40 features (real numbers). The output Y has 10 ordered values (0, 1, 2, ... 10) which represent the level of anxiety (it's the answer of a questionnaire).
The output are in on a interval scale and are not just ordinal.
Due to the nature of the problem, me and my collegues have considered the possibility of reducing the output, from a range of 10 values to 3 different output. Something like 1, 2, 3 or for example "low", "medium", "high".
Edit: Basically I want to reduce the size of the interval of the output.
My questions are:


*

*Once the discretization (reduction of number of labels) has been performed, should we still use regression algorithms, or since the low number of output we are in a classification scenario?

*Which is the best way to reduce the number of labels?

*Would it have sense to continue using regression and then categorize the prediction output in three different levels?

 A: Never reduce the number of levels of an ordered or continuous variable.  This results in loss of power and precision and increase in instability.  For your problem an ordinal regression model such as the proportional odds model is called for. To handle the fact that you have too many candidate predictors for your effective sample size, use one of the following approaches:


*

*penalized maximum likelihood estimation to effectively reduce the number of parameters being estimated

*data reduction (unsupervised learning) that is masked to Y; here you collapse the predictor space using things like variable clustering, principal components, or redundancy analysis.  This usually helps in interpretation also.


See my course notes for more details.
Note that your output variable Y is probably not interval scaled.
A: Here are my answers to your questions:


*

*This depends on how you define discrete levels. I would opt for regression followed by discretization if there are strict and crisp differences between new levels of the output. However, if new labels are fuzzy, so that it makes more sense to talk about probability of being at "low" (or "medium" or "high") level, I would use classification algorithms.

*This depends on the problem that you are solving. You should figure out the most reasonable way to define "low", "medium", and "high" values such that it makes sense in the underlying theory of the problem (bio-signals in your case). You can find two critical values, $\alpha$ and $\beta$, such that the output is considered low if it is less than $\alpha$, normal if it is between $\alpha$ and $\beta$, and high if it is greater then $\beta$. You can do this by constructing distribution of outputs on the current scale ($1,2,\ldots,10$), and then determine $\alpha$ and $\beta$ as percentiles (for example, let $\alpha$ be the 10th percentile and $\beta$ the 90th). It is hard to give any solid advice on actual values of $\alpha$ and $\beta$ without knowing the nature of data. 
Maybe you can even use fuzzy set theory to define membership functions of "low", "medium" and "high", and then use them to determine output label of a concrete example. There are several alternatives. :)

*In my opinion, it would have sense.

