# Factor analysis and Dimensional Reduction difference

In exploratory factor analysis, we can clearly see the weightage of the factors on different features. Following this, we can understand the extent to which factors have an influence on the features. I understand this as factor analysis.

In other techniques, be it supervised like PCA or unsupervised like LDA, is the aforesaid possible? For example in PCA, the factors get boiled down to an equal number of features, and depending on the variance explained we extract it out. Similarly for LDA, the number of classes minus one, factor gets extracted. It essentially reduces the dimension, calling it a dimensional reduction process.

So does this mean, dimensional reduction techniques like PCA and LDA lack the explanatory components on determining which features are influencing the factors? If so, do there exist any other techniques which work in a similar way to exploratory factor analysis, rather than just dimensionally reducing the data?

In fact, PCA can be understood as a special case of FA. Both FA and PCA presume the same model: $$\mathbf{y} = L\mathbf{x} + \boldsymbol{\epsilon}$$ where $$L\in\mathbb R^{n\times d}$$, $$\mathbf{x}\sim N(0, \mathbf 1), \mathbf x\in\mathbb R^d$$, with $$d$$ the dimension of the dimension-reduced linear subspace, $$\mathbf y\in\mathbb R^n$$, with $$n$$ is the number of your features, and $$\boldsymbol\epsilon\in\mathbb R^n$$ is the noise. The only difference between FA and PCA is the distribution of the noise: \begin{align} \mbox{PCA}&: \boldsymbol{\epsilon_{PCA}}\sim N(\mathbf 0, \sigma^2\mathbf 1)\\ \mbox{FA}&: \boldsymbol{\epsilon_{FA}}\sim N(\mathbf 0, \mathbf D). \end{align} where $$\mathbf D$$ is a diagonal matrix $$\mathbf D = diag(\sigma_1^2,\ldots,\sigma_n^2)$$.