It is said that supervised is different from unsupervised learning due to the presence of "supervisory signals" aka labels.
However, in both cases we have a loss function. Isn't the loss a kind of "supervisory signal" that guides the optimization and thus the learning process?
You're right in noting that both supervised and unsupervised learning usually involve loss functions guiding the optimization process. However, in unsupervised learning there are no explicit supervisory signals as labels provided, so the loss functions coming from the data's inherent structure are implicit, more abstract, and often more versatile. They often take many different forms such as similarity (contrastive/triplet learning), cumulative in-group variance (K-Means Clustering), reconstruction error with possible regularization (PCA, VAE, DDPM), minimax loss (GAN), etc, in addition to the common MSE or CE loss used extensively in supervised learning.
