Cluster data into categories; train one classifier per category Let me first give you a hypothetical example to make things more clear: 
Let's say your task is to classify art as either professional or amateur work, based on image data. You extracted 100 features from each artwork using image processing. The artworks fall into different genres (like sketch, abstract paintings, portraits, still-life, oil painting, watercolor painting etc.). 
These genres are responsible for most of the variance in the features. The genres (or even their number) are not known to you a priori. Indeed, there may be different ways to cluster this data into genres/categories (e.g. based on subject, painting technique, colors used).
The class distribution (professional, amateur) varies per genre. For instance most amateur works in your dataset could be still-life. So if you would use a linear classifier it would basically just classify everything as still-life or not still-life.

Is there a model that is known to work well for a dataset like this?
The obvious approach would be to throw a non-linear classifier (like kernel svm, decision forest) at it and hope that it can deal with the problem. However, that may work less well then a model that can make use of this property of your dataset explicitly.
Another approach would be to cluster the data first, then train one classifier per cluster. However, that may lead to clusters that are not optimal for classification.
So is there a model that is know to work well for problems like this on (bigish) real world datasets? Maybe modeling the problem probabilistically as 
$$P(\text{professional}) = \frac{P(\text{genre}) \times P(\text{professional}|\text{genre})} { P(\text{class} | \text{professional})}$$
and applying EM, starting from with a random class alignment. Or modeling it as a neural network, that has this structure hardwired and applying backpropagation.
 A: What you have described here is in fact the case with every classification problem. You can always find some subsets of the data, that are simplier to classify then the original problem. There are dozens off approaches here, as it was previously mentioned - it is in fact a general case. If you are looking for something that actually incorporates this kind of structure in the "meaningful" way (easy to understand) you could give Hierarchical Classifier with Overlapping Classes (HCOC) a try: http://dl.acm.org/citation.cfm?id=1244576 . This model tries to build the tree like decision tree and alternates between classifing step and clustering your data. On the other hand, if you know exactly what kinds of pictures you can expect (which forms some statistically different classes) then you should simply build a classifiers (rule based, or learned) to classify your data into these classes $cl(x)=c_i$ and then train independent classifiers $cl_{c_i}(x)=output$, and then use the pipeline $cl_{cl(x)}(x)$ as a main model. If you choose to do so, you could build a good classifier, but if your data is actually big then you seem to fall into the on-going for many many years discussion of "isn't it better to include experts knowledge instead of learning everything?" and moden machine learning is closer to answer no, it is not.It is better to use better models on the whole data, without human processing, feature engineering and pre-clustering, because statistical models are simply better then we are. This is somehow the core concept of deep learning, which shows great results with little (or none) preprocessing. 
To sum up:


*

*HCOC model does something closely related to your idea

*The "naive" approach can be done, yet it is rather suited for the small datasets, where expert knowledge (which you propose to implement) can overcome risks of overfitting and/or biasing

*For big data, it would be more reasonable to look for better model, then better preprocessing - like deep learning

A: If the genres are not known apriori, and assuming the distribution of genres in the dataset is similar to that of your real application, then what is the problem with your model taking the genre into account when performing classification? In the extreme case, if it classifies all "still-life" work as "amateur", is that a bad outcome?
I can only see a problem if your dataset contains a biased sample. Then your model can end up being biased as well.
