Automatic text quality grading I came up with an idea to use machine learning for automatic grading of topic-specific texts. 
More specifically, I will first use normal text classification techniques to sort all candidate texts into topics. Then, I want to be able to judge the quality of texts in specific topics:
For instance, news articles in different topics (Technology, Sports, International, Business, etc). Firstly, every article will be sorted into one topic. And what I want to archive then is to automatically grade this 'Technology' article based on the range of 1-10 or good/poor.
The criteria for such scoring scheme may be:


*

*Elaboration (rich in details)

*Coverage of all key words

*Length?


(It is actually a little bit hard for me to quantify my criteria. The grading criteria here can be very similar to those used when human beings instinctively and quickly judge quality of two articles on a same story.)
However, as I reckon, text classification based on 'bag of words' may not be able to perform this task well. (Or will it?)
I believe such task is well-researched, and even it is a whole field rather than text classification. But I haven't figured out how to do this by searching around. Could you please refer me to related techniques/discussions/names or give me some hints if text classification can handle that.
Many thanks in advance.
 A: There are two basic components, technical analysis (grammar, sentence length, etc) and machine learning/statistical analysis.
The best (IMHO) papers on this are those involving automated essay grading.  It's a little dated, but this paper covers the techniques employed by all of the major vendors.
ETS (the people behind the TOEFL and the GRE) have put a LOT of work into this area, and they are surprisingly open about their research.  The wall they run up against is that of length and specificity.  They can only grade papers (with a high degree of accuracy) that are short and have specific prompts.
The biggest problem I see is that ALL of these grading facilities are based on machine learning.  Even ETS uses their graders as backups for human graders; they pay 1 human to grade and if their grade doesn't match the ML grade, the essay is handed to another human grader.  It essentially cuts their costs in half but it doesn't replace human graders.
If you really want to try analysis on ad-hoc text you will probably need to use the API of someone that has access to a lot of unstructured text, like OpenCalais or AlchemyAPI or even Zamenta for the categorizing and then roll your own quality analysis.  However, that's a bit out of my pay range.
update Jonny's answer is really not getting enough credit.  If you are really concerned about content quality, analyzing network connections and other measurements of human behavior are REALLY valuable.  Again, I am no expert in this field!
It would help if you could tell us what our accuracy/validity requirements are.
A: Actually if you want to judge the quality of works, you may want to approach this with something like Google Pagerank. So for example if a particular paper is footnoted or referenced in many other papers then there is a greater chance that it is a well respected paper than a paper that is never footnoted.
Another factor which I believe would have a high correlation with quality would be the author(s). While not perfect, it is probably reasonable to assume that works by Nobel laureates within their area of expertise would tend to be of higher quality than the first work of someone who has never published before.
Third you could consider the publisher. Articles from peer reviewed journals should tend to be of higher quality than those from tabloids. 
The problem with some of the items you mentioned are that they are either irrelevant to quality (length for example, by that measure the Lincoln's Gettysburg address would fail) or virtually impossible to score. 
Suppose I wrote a paper about the solar system and gave the distance to the moon as 500,000 miles and said the sun was at half that distance with a temperature of 27 degrees kelvin. Would your program be able to verify or disprove those "facts"? It would score high on elaboration (lots of details, all completely wrong). As for keywords, we all know about keyword stuffing.
Some things you could easily score with a program however, such as reading level (fog index). 
A: More a comment than an answer, but I suspect that trying to capture the quality of a particular document in a single (non-adaptive) score is the canonical example of YMMV.
For example, in a technical context users might consider quality to mean in-depth treatment of a specific problem, a survey of different techniques, position statements from experts, or any of various other measures of quality, each depending on how they want to use the information they get from the document. Furthermore, depending on the users' levels of technical sophistication, different users can be expected to rate the same documents differently, even if they have similar aims.
On the other hand this is exactly the sort of problem faced (and probably in the current state of the art partly solved) by recommender systems (Netflix ratings, Amazon recommendations, and so on). I'm no expert on these sorts of approaches, but no doubt others here are. For your problem I'd probably start my literature search in that domain.
