Mixing unsupervised and supervised learning Is it a good idea to mix supervised and unsupervised learning together? I'm trying to predict some sales data given some data on hand, so I think a regression is the best way to go.  However, I'm also not sure on what is important information or not.  So I was thinking that I start off treating this as an unsupervised learning problem (maybe use clustering?) to try and weed out any unnecessary attributes and then treat the remaining data as a regression problem. Or am I approaching this problem completely wrong?
I'm pretty new to machine learning, so I'm sorry if this is a strange question to ask.
 A: From a definitional sense, there is no such thing as "mixing unsupervised learning and supervised learning" since any problem for which you have target variables is by definition supervised learning.  When you don't have target variables it's called unsupervised learning.  (See the introduction from C. M. Bishop, Pattern recognition and machine learning, 1st ed. 20. Springer, Oct. 2006.)
This is complicated because any supervised learning algorithm can be adapted into an unsupervised learning algorithm by letting the targets be the inputs.  E.g., autoencoders are unsupervised neural networks.  And any unsupervised learning algorithm can be adapted into a supervised one by letting targets be inputs as was done for RBMs (figure 4 here: Hinton, G. E. (2007). To Recognize Shapes, First Learn to Generate images. Progress in Brain Research, 165, 535–547.)
But in general, I think there is a clear difference between what typical unsupervised learning algorithms do well, and what typical supervised learning algorithms do well.  Unsupervised learning algorithms create features from inputs: sometimes called discovery.  Supervised learning algorithms learn mappings from features to targets: sometimes called learning  (it's a bad name!)
If you have targets, I think it's better to apply supervised learning techniques.  Even Boltzmann machines have a final training phase using gradient descent.
But if your input features are in an inconvenient space, or are highly correlated (in some space) then transforming them using unsupervised learning is going to help a lot.
If both points apply — you have targets and your inputs are correlated — then you should combine both techniques.
A: I have no problem mixing "supervised" and "unsupervised" techniques. What regression modeling assumptions are violated by doing so? For instance, by flipping the mode of the data matrix under analysis from cluster solutions to PCA yields another "unsupervised" technique -- which is like regression without a dependent variable. Many people prefer to run PCAs as an exploratory step intended to weed out redundancy among predictors. So, depending on how you use the results, it can provide insight into the variable selection process, contributing to a "supervised" predictive model. 
A: I personally do not recommend mixing unsupervised and supervised learning techniques. The simple reason for that is that the label can only aid you in your learning problem. 
For example in your case you want to do feature selection as you are unsure about what features are best. Well then how about supervised feature selection? There are plenty of supervised feature selection methods available, of which the simplest investigates the correlation between each feature and your label. 
Have a look at 
https://en.wikipedia.org/wiki/Feature_selection#Main_principles to start.
In the end I do want to emphasise that unsupervised feature selection techniques such as clustering can produce nice results. However, as a general rule of thumb: more accurate information yields better performance. 
