I have a dataset of 5000 observations and 100 features. Some people recommend me to use ANOVA and Chi-square test to drop some independent variables that are independent of my dependent variable. However, my concern is my data is not obtained from an experiment but from observation. I cannot conclude from ANOVA and Chi-square test that some of the independent variables are indeed not related to my dependent variable. Can you guys please tell me whether I am right? Thanks
You are wrong when you say that
I cannot conclude from ANOVA and Chi-square test that some of the independent variables are indeed not related to my dependent variable.
that's not dependent on the data being from an experiment (however, with observational data you have to be very careful about making causal statements).
But doing variable selection this way (which is called bivariate screening) is not a good method (although it is very common). There are lots of threads here on variable selection; if you want a whole book, I highly recommend Regression Modelling Strategies by @FrankHarrell.