I have what seem like a silly question about dimensionality reduction.
I am just learning about this now on my own, and most of the information I have found is centered around reducing dimension by figuring out whether or not linear dependence is present. Upon learning about dimensionality reduction, what I sort of assumed was that one method would be to run a regression and figure out whether or not your variables were statistically significant or not. If they were not, then they could be eliminated, and this as well would reduce dimension. Is this also a correct method? Am I thinking about this the right way?
Please keep in mind that I am an undergraduate and have only taken introductory courses on regression, linear algebra, statistics, etc but I am interested in these topics and just trying to develop some intuition for different methods and when they are appropriate.