If I am correct, "unsupervised classification" is same as clustering. Then is there "unsupervised regression"? Thanks!
I've never encountered this term before. I am unsure whether it would spread light or darkness within either realm of statistics: those being machine learning (where supervised and unsupervised distinctions are central to problem solving) and inferential statistics (where regression, confirmatory analysis, and NHSTs are most often employed).
Where those two philosophies overlap, the majority of regression and associated terminology is thrown around in a strictly supervised setting. However, I think many existing concepts in unsupervised learning are closely related to regression based approaches, especially when you naively iterate over each class or feature as an outcome and pool the results. An example of this is the PCA and bivariate correlation analysis. By applying best subset regression iteratively over a number of variables, you can do a very complex sort of network estimation, as is assumed in structural equation modeling (strictly in the EFA sense). This, to me, seems like an unsupervised learning problem with regression.
However, regression parameter estimates are not reflexive. For simple linear regression, regressing $Y$ upon $X$ will give you different results, different inference, and different estimates (not even inverse necessarily), than $X$ upon $Y$. In my mind, this lack of commutativity makes most naive regression applications ineligible for unsupervised learning problems.
The closest thing I can think of is a little black magic that stirred people up when it was announced a few years ago, but I don't believe it gained any real traction in the community. The authors developed a statistic they called the "Maximal Information Coefficient (MIC)." The general idea behind their method is to take highly dimensional data, plot each variable against every other variable in pairs, and then apply an interesting window-binning algorithm to each plot (which calculates the MIC for those two variables) to determine if there is potentially a relationship between the two variables. The technique is supposed to be robust at identifying arbitrarily structured relationships, not just linear.
The technique targets pairs of variables, but I'm sure it could be extended to investigate multivariate relationships. The main problem would be that you'd have to run the technique on significantly more combinations of variables as you allow for permutations of more and more variables. I imagine it probably takes some time just with pairs: attempting to use this on even remotely high dimensional data and considering more complex relationships than pairs of variables would become intractable fast.
Reference the paper Detecting Novel Associations in Large Datasets (2011)
This question came to my mind while researching the difference between supervised and unsupervised methods. Coming from an econometric background I prefer to think in models, which slowed my understanding as most machine learning literature I encountered focuses on methods.
What I have found thus far is that a strict distinction should be made between
clustering (unsupervised) versus
classification (supervised). The continuous analogy of the relation between these model designs would be
principal component analysis (unsupervised) versus
linear regression (supervised).
However, I would argue that the relation between clustering and classification is purely coincidental; it exists only when we interpret both model designs as describing a geometric relation, which I find unneccesarily restrictive. All unsupervised methods that I know of (k-means, elastic map algorithms such as kohonen/neural gas, DBSCAN, PCA) can also be interpreted as latent variable models. In the case of clustering methods, this would amount to viewing belonging to a cluster as being in a state, which can be coded as a latent variable model by introducing state dummies.
Given the interpretation as latent variable models, you are free to specify any, possibly nonlinear, model that describes your features in terms of continuous latent variables.