Regression model for unbalanced count variable I am trying to predict a count variable that ranges between 0 and 10 and  60 out of the  94 examples have the value 4. 
I rejected building a classification model, since that would ignore the inherent order of the data. Also, regression models are defined for continuous variables, so generalized linear models are more appropriate in my case.  
The main problem of my data is the abundance of 4 values, which means that the rest are underrepresented. This leads to a model that usually predicts 4 and misses all other values. What should my approach be?
 A: While you have not sufficiently described your approach, I can guess at the problem: its cause and its solution. 
The problem is that you are implicitly using a classification model, despite claiming otherwise. The classification is one which applies the mode. You will find it is the default approach when applying neural nets or log linear models in the nnet package in R and the polr function in MASS. If you look at the predict methods for these fit objects, you will find that probabilistic predictions can be created instead. Mode classifications from probability models perform poorly because so many categorical or ordinal variables are multimodal.
If you augment the predictions to return class probabilities, you might stop here and consider probabilistic evaluations of predictive accuracy instead. For predicting binary outcomes, we sometimes aren't interested in developing a classifier, but summarize the performance of the model of a range of possible classifications. The model can correctly tell me that 70% of responses are 4, 20% are 10 while the remaining 10% are 0. That would be a good model. Predicting individual level responses may not be possible for a lack of features. You can inspect a weighted kappa to evaluate the agreement between the observed and expected response frequencies. Obtaining predictions may be as easy as simulating data according to the predicted frequencies.
A different approach is building a decision tree. I have no idea why we would expect or believe 4 to be an inherently interesting value, but since it is an important structure in your data, you may look at the probability model for observing a response of 4 versus other responses. (A saturated log-linear model does this independently). 
