0
votes
1answer
44 views

Repeatedly measuring accuracy against the hold out set

I have an iterative document classification task, corpus size = 300,000 documents. The labels are binary valued (yes/no). I wanted to know whether the following methodology is valid. The assumption is ...
0
votes
1answer
131 views

Accuracy rate in naive Bayes classification

I am trying to use a naive Bayes classification technique to predict fraudsters (Caller). My training set of 138 instances has 5 columns viz. ...
2
votes
1answer
150 views

What is the best way to compute classifier performance metrics given a confusion matrix?

Let's say I compute a confusion matrix, in the sense defined here: http://www.gabormelli.com/RKB/Confusion_Matrix I could easily compute the number of True Negatives (TN), True Positives (TP), False ...
4
votes
1answer
215 views

Name of mean absolute error analogue to Brier score?

Yesterday's question Determine accuracy of model which estimates probability of event got me curious about probability scoring. The Brier score $$\frac{1}{N}\sum\limits _{i=1}^{N}(prediction_i - ...
2
votes
0answers
91 views

Validation of data partitioning

I've some numerical sequence of data monitored over time $v(i)$, which belongs to one of two classes, 0 or 1. I monitor the variable several times, obtaining a sequence associated to 0 or 1. I use ...