I am trying my to built a model that predicts whether or not customers will churn, using a dataset with 7000 instances (rows) and 20 features. I am using WEKA and experimenting with a J48 Decision tree.
What indicates that my model is good at predicting? And when is it good enough for practical use?
ZeroR (Baseline): 73%
J48 5-folded cross validation: 78%
Random Forest 5-folded cross validation: 79%
The way I interpret this is that my model is better than purely guessing, but not good enough for practical use. How would you interpret this? If you have tips to improve accuracy please share.