What is "Prediction Accuracy (AUC)", and how is it the number conducted in Machine Learning? Here is the link in question: http://applymagicsauce.com/documentation.html
When the Cambridge University Psychometric Center's "Apply Magic Sauce" defines how their Prediction Accuracy (AUC) system works, this is what they say:

Prediction accuracy is expressed as the correlation between the AMS prediction and the actual score. Accuracy of 1 indicates a perfect accuracy, whereas the accuracy of 0 indicates a random guess.

What is "the actual score"? In simple english, how is it calculated? I'm relatively new to these concepts, and I would just like to know how accuracy rates are calculated in machine learning.
When the documentation page states:

The model was build (and the accuracy validated) using a sample of 98,000 people.

What exactly do they mean? How was it validated, and what is the simplest way of doing so with a text classification system?
 A: On the page linked, there are two metrics used to measure predictive accuracy of the algorithm. 
1.The first is the usual Area Under the Curve (AUC) of the Receiver Operator Characteristic. As explained in the comments, AUC ranges from 0.5 to 1, with 1 being perfect classification and 0.5 being no better than luck.  

Since there are only two outcomes, the algorithm can either classify positive correctly, or incorrectly.  Similar to flipping a coin, chance will predict correctly 50% of the time, giving 0.5 AUC a "worthless" accuracy.  As you increase from 0.5 to 1, your algorithm gets better and better at correctly classifying outcomes.  
These algorithms are usually trained on a training set to tune parameters and tested on a test set to determine their accuracy and applicability. The AUC is found on the second table on the page.
2.The second of the metrics used is simply the correlation between the predicted outcome and the actual outcome.  This was used because the outcomes were not binary; they were linear, and thus classification was inappropriate. 
This has the range 0 to 1, as there can be 0 correlation between the predicted and actual outcomes. The correlation is found on the first "linear" table on the page. 
As for the second part of your question, the 98,000 person sample that they mentioned most probably had known outcomes.  For example, if your algorithm predicted a case (1), you could compare it to the actual known data to see if your algorithm predicted a true positive (TP) or a false positive (FP).  This is how they measure the accuracy.  
As for the text classification system, you would need to provide more details, perhaps a separate question for that might be appropriate. 
