I am working on a problem to predict a value between 0 and 1. The data is skewed, so that only 18% of the values are greater than 0.
I implemented a machine learning system, and what is happening is that the lowest error rate on the validation set (18%) is found when all predictions are 0.
So, I wanted to implement precision, recall and F-Scores, to find ways of more accurately measuring my results (instead of error rate and MSE). But precision, recall and F-Scores seem to apply to classification problems.
What is a better approach to measuring the results in this case? Is there a good way to use precision, recall and F-Scores?
EDIT: To clarify, I think it is a regression problem and should output a continuous value between 0 and 1. It is a recommendation engine using Collaborative Filtering. I am creating a matrix between users and items, and minimizing the MSE.