Here is the link in question: http://applymagicsauce.com/documentation.html
When the Cambridge University Psychometric Center's "Apply Magic Sauce" defines how their
Prediction Accuracy (AUC) system works, this is what they say:
Prediction accuracy is expressed as the correlation between the AMS prediction and the actual score. Accuracy of 1 indicates a perfect accuracy, whereas the accuracy of 0 indicates a random guess.
What is "the actual score"? In simple english, how is it calculated? I'm relatively new to these concepts, and I would just like to know how accuracy rates are calculated in machine learning.
When the documentation page states:
The model was build (and the accuracy validated) using a sample of 98,000 people.
What exactly do they mean? How was it validated, and what is the simplest way of doing so with a text classification system?