I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are doing the same thing in both of them?
The difference is that the set of features made by feature selection must be a subset of the original set of features, and the set made by dimensionality reduction doesn't have to (for instance PCA reduces dimensionality by making new synthetic features from linear combination of the original ones, and then discarding the less important ones).
This way feature selection is a special case of dimensionality reduction.
Feature selection works on Variance and Dimensionality reduction works on Eigen value and Eigen vector.
In feature selection we are actually working on attributes and leave the attributes based on variance but in case of dimensionality reduction we create new dimension based on covariances.
Hope my answer helps you thanks for asking the question.