Datapoint Classification Accuracy I am interested in finding ways to quantify the certainty of correct classificaions for single datapoints. This is interesting for me since for clinical studies where we for instance would classify subjects with an without autism, it would be hugely helpfull to add to the diagnosis a certainty about the diagnosis.
So far I know about methods which assume a certain distribution for the data like naive Bayes Classifiers and the like. Generally, I am interested about hearing about all the approaches there are, and especially about quantifications which are interpretable as well as not specific to a unique trained classifier.
Every hint is very welcome
 A: If I understood the question correctly, you have a probabilistic prediction for a classification, that is you make a prediction that the new patient will be autisitc with y% probability and and not autistic with probability (1-y)%.
In this case you may want to start with the concept of "scoring rules" http://en.wikipedia.org/wiki/Scoring_rule which is the term used for the evaluation of how well a probabilistic prediction (in classification) is. A another usual term is "calibration". 
A: If I understood the question correctly, you are looking for the posterior probability for the data points. So, in principle you must be able to get this information if you used a probabilistic model for your classification be it naive Bayes or logistic or any simple regression. 
If this is what you are looking for, depending on what platform you program it is possible to obtain it. Let me assume that you use R: 
In that case, you can first fit a nauve bayes model using 'naiveBayes' function in the 'e1701' package. And then using the 'predict' function, you calculate the posterior probablities of a point belonging to a particular class given the data. Here is a small pseudo code: 
 library(e1701)
model=naiveBayes(Class ~ ., data = X) ## Here let X be a dataframe with one column named Class which is the class of the samples. 
predict(model, X, type = "raw")

Hope this helps... depending on what model you use, the predict function will take different arguments. Information about this can be found in the 'stats' package. 
