Creating a disease severity score/index I am trying to design a numerical scale which would describe the severity of a certain disease (in this particular case anaphylaxis). I have a set of clinical symptoms and a database of patients who had anaphylaxis and had their symptoms described. 
I would like to come up with a numerical value for each patient resembling the severity of their anaphylactic event.
I could assign numbers to symptoms according to their severity but this would be highly subjective (for example rash 1 point, cardiac arrest 20 points)
My question is the following: How may I validate if this what I have designed is good or  bad? How could I compare two models of this index? Are there any tools or methods that could help me with this subject ? Is there a manual or a book I could read? 
By severity I mean the impact the anaphylactic event has on a patient's health. If the patient dies then the severity is the greatest.
If it is only a mild reaction like a rash/cough/rhinitis/ then the index should be low.
 A: The method described in this blog post seems well suited to your needs.  You are seeking a "severity index" which is equivalent to the "performance index" in the blog post.
There are two design decisions associated with your problem. First, what is your set of "performance hypotheses"?  i.e. How does the evidence from each of the symptoms relate to possible values in the performance index?  
The second design decision relates to the scaling and resolution for index of severity.  An interval scaled index presumes that the difference between a 2 and a 4 is the same as the difference between a 4 and a 6 (on a 10 point scale).  You also need to test the limits of your scale and the regions near each extreme.
In other words, you need to both calibrate your scale and also support the claim that there is a meaningful and equal difference between successive values on the scale.  This implies that you should not use a scale whose resolution is not supported by meaningful differences.  In simple language, don't pick a 1 to 100 scale if you can only meaningfully differentiate 5 levels or states.
You asked the question: "How may I validate if this what I have designed is good or bad? How could I compare two models of this index?"
There are two ways to test your models -- 1) internal validity and 2) external validity.  Internal validity tests are those that look for consistency between rules, and consistency between definitions and outcomes.  These are basically 'bugs'.  External validity tests look for consistency between model results and out-of-band results, starting from the same inputs.
Given your situation, I'd say that external validity tests will be most important.  Essentially, for each performance hypothesis you define, you should look for one or more external (out-of-band) tests for that hypothesis.
For example, you might have some symptoms that are very common across outcomes, and thus not very informative regarding the severity index.  Likewise, you might have some symptoms that are rare and mostly associated with specific levels of severity, but also might be noisy (i.e. have high error rate, or high uncertainty).  You should look for external tests for these hypotheses from the existing body of research.
