Is there a more precise definition of generative models? Intuitively, a generative model is one that we can generate good data from.  What I'm looking for is a formal definition of "good data."  For example, for any classification model I could generate data by simply sampling really close to the decision boundary.  But this is not "good data" because the data will probably be from a very different distribution as the original data.
Is there a more precise definition of what constitutes a generative model?
 A: Test set Log-Likelihood is an objective measure of model performance.  It can be intractable or difficult to evaluate for some models.  This is equivalent to minimizing the KL divergence between the test data distribution and the model (remember that KL is not symmetric).  
It is not necessarily a good way to evaluate a generative model in general, though.  
It is common in deep learning generative modeling of image datasets to visualize samples from the model as a qualitative way of evaluating performance.  For large datasets, it can be difficult to tell how much generalization is occurring, however.  One thing that can help is looking at nearest neighbors of the samples in the dataset.  
It would also be interesting to look at other divergences as a way of evaluating.  There are things like Fischer Information metric, and these http://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence#Symmetrised_divergence to check out.  But it is not possible to calculate these in a meaningful way when all we do not actually have the data distribution, only example data points (as far as I am aware).  
A: You can generate Y data, conditional on X, for any classifier.
But a generative model allows you to probabilistically generate X, Y pairs.  See http://ai.stanford.edu/~ang/papers/nips01-discriminativegenerative.pdf
