Machine learning *why* instances are a certain classification I know machine learning algorithms are good at determining/predicting what an instance is (classification and such) or how instances are related. Are there any algorithms that, given classified training data and classified test data, can describe (in some way, maybe pointing out attributes) why an test instance is classified a particular way.
An example might help as well. Say we're dealing with a binary-class dataset:
    Class    A    B    C    D    ...
1   T        0.5  0.7  0.3  0.9  ...
2   T        0.6  0.2  0.4  0.8  ...
3   T        0.5  0.6  0.6  0.7  ...
4   F        0.9  0.2  0.2  0.5  ...
5   F        0.9  0.8  0.4  0.5  ...
...  

Now we want to train a model, given training data of this type, such that we can give a test instance and ask, "Why, or why not, is this instance class=T? What variables have the most significant impact on the classification?
So I could then give a test instance with a classification given:
    Class    A    B    C    D    ...
X   T        0.4  0.9  0.8  0.9  ...

The algorithm could then respond, maybe indicating specific attributes which seem to be indicators of distinction between the two classes or something along those lines. For example: "Instance X would tend to be classified as T because value A tends to be around 0.5 and D tends to be around 0.9." Or alternatively there may be no good reason for X to be classified as T, then the algorithm would either be inconclusive or have low confidence measure on its results.
I'm not sure if anything like this exists, but have searched around and thought it might be useful to ask here. Any help (for good or worse) is greatly appreciated. Thank you.
 A: Decision tree learning produces models that are very easy to interpret.  You can just trace an input's path down the tree and see exactly why it ended up in the class it landed in.  That direct interpretability is actually a key strength of decision trees.  Similarly, random forests, 
 which are made of decision trees, have a similar benefit.  But interpretation is less direct because there are many trees to consider for a single input.
More complex, nonlinear, or highly dimensional models (like neural networks) still have rules that govern input to output relationships.  Those rules are usually even deterministic, so you can trace the inputs in the same way as with decision trees.  The problem arises from interpreting that information in a meaningful way.
A toy example about house price prediction might help demonstrate what I mean.


*

*A decision tree can directly tell you that a house ended up in the 500K-750K bucket because it has 2 bathrooms, 2 bedrooms, between 1200 and 1400 square feet, and is newer than 5 years old.

*A neural net making the same prediction will directly tell you that 100 inputs to 50 tanh functions across 3 layers gave a softmax output that corresponds to 500K-750K.
The latter is much more difficult to interpret, but not a lost cause.  
Look up model/prediction interpretation for the type of model you're interested in to see how others have approached the problem.
A: You are talking about variable or feature selection, of which the literature is vast. 
Various methods exist, which can be broadly characterized in three categories: 


*

*Filter methods: these methods typically use univariate statistics to filter out or select features in function of your target variable. 

*Wrapper methods: these methods incrementally add/remove features which are best/worst performing. 

*Embedded methods: these methods instantly give you a score along with your prediction. As the answer above, Random Forests for example, will be able to give you along with your trained model an output which is called variable importances.


An overview of review papers concerning the topic of variable selection can be found here. 
